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Abstract 

Almost  as  long  as  there  have  been  user  interfaces,  there  have  been  special  software  systems  and  tools  to 
help  design  and  implement  the  user  interface  software.  Many  of  these  tools  have  demonstrated  significant 
productivity  gains  for  programmers,  and  have  become  important  commercial  producLs.  Others  have 
proven  less  successful  at  supporting  the  kinds  of  user  interfaces  people  want  to  build.  This  article 
discusses  the  different  kinds  of  user  interface  software  tools,  and  investigates  why  .some  approaches  have 
worked  and  others  have  not.  Many  examples  of  commercial  and  rc.scarch  systems  arc  included.  Finally, 
current  research  directions  and  open  i.ssucs  in  the  field  are  di.scu.s.scd. 
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1.  Introduction 

User  interface  software  is  often  large,  complex  and  difncuii  to  implement,  debug,  and  modify  One 
study  found  that  an  average  of  48%  of  the  code  of  applications  is  devoted  to  the  user  interface,  and  that 
about  S0%  of  the  imptementation  time  is  devoted  to  implemenung  the  user  interlace  portion  [57 1.  As 
interfaces  become  exsicr  to  use.  they  become  harder  to  create  |f>4|  Today,  direct  manipulation  interfaces 
(also  called  ■‘Gl'ls  ’  for  Graphical  User  Interfaces i  anr  almost  universal  one  study  found  that 
of  all  software  development  on  Unix  involved  a  GUI  [126.  p  SI))  ITiese  interfaces  a-quia*  that  the 
programmer  deal  with  elafxvrate  graphics,  multiple  ways  for  giving  the  same  command,  multiple 
a.syTichiTXK>us  input  devices  (u.sually  a  keyboard  and  a  locator  or  pointing  device  such  as  a  mouse),  a 

mode  free  interface  where  the  user  can  give  any  commarxl  at  virtually  any  time,  and  rapid  semantic 
feedback  "  where  determining  the  appropnaie  response  to  u.scr  actions  requires  speciali/ed  inlormation 
about  the  objects  in  the  program  Tomorrow's  user  interlaces  will  pnivide  speech  and  gesture 
recognition,  intelligent  agents  and  mtegrated  multi-media,  and  will  probably  be  even  more  difficult  to 
create.  Furthetmorc.  because  user  interface  design  is  so  difficult,  the  only  reliable  way  to  get  good 
interfaces  is  to  iteratively  re -design  (and  ihcrcfore  re-impicment)  ihe  interfaces  after  user-testing,  which 
makes  the  implementation  task  even  harder 

Fortunately,  there  has  been  significant  progress  in  software  tools  to  help  with  creating  user  interfaces, 
and  today,  virtually  all  user  interface  software  is  created  using  tools  that  make  the  implementation  easier. 
For  example,  the  MacApp  system  from  Apple  has  been  reported  to  reduce  development  time  by  a  factor 
of  four  or  five  [921.  A  study  commissiorKd  by  NeXT  claims  that  the  average  application  programmed 
using  the  NeXTStep  environment  wrote  83%  fewer  lines  of  code  and  took  one -half  the  time  compared  to 
applications  written  using  less  advanced  tools,  and  some  applications  were  completed  in  onc-lcnlh  the 
time  [7]. 

Furthermore,  user  interface  tools  are  a  major  business.  In  the  Unix  market  alone,  over  US$133  million 
of  tools  were  sold  in  1993,  which  is  about  50,000  licenses  [126j.  This  is  a  64%  increase  over  1992. 
Forrester  Research  claims  that  the  total  market  for  UI  software  tools  on  all  platforms,  including  “vertical 
tools”  which  include  database  and  user  interface  construction  tools,  will  be  130,000  developers 
generating  US$400  million  in  revenue.  They  estimate  that  this  will  double  each  year,  growing  to  700,000 
developers  and  $1.2  billion  by  1996  [16]. 

Mark  Manner  from  the  Meta  Group  market  research  firm  says  that  the  user  interface  tool  market  is 
about  to  explode  (24).  Whereas  the  “first  generation”  of  commercial  tools  were  not  fully  graphical  or 
were  not  sufficiently  powerful,  this  is  no  longer  tnic  for  today’s  tools.  Furthermore,  prices  for  tools  have 
dropped  significantly,  and  fees  for  run-times  have  been  mostly  eliminated  (so  that  designers  do  not  have 
to  pay  the  tool  creator  for  products  created  using  the  tools).  For  the  future,  there  is  still  a  tremendous 
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opportunity  for  good  tools,  espcc  n  niche  arexs  like  multimedia,  distributed  systems,  and 
geographical  information  systems. 

This  article  surveys  user  interface  software  tools,  and  explains  the  different  types  and  classifications. 
There  have  been  many  previous  surveys  on  this  topic  (49,  25],  but  since  this  is  a  fast-changing  field,  a 
new  one  seemed  in  order.  In  addition,  this  article  takes  a  broader  approach  and  includes  more 
components  of  user  interface  .software,  including  windowing  systems.  However,  it  is  now  impossible  to 
discuss  dll  user  interface  tools,  since  there  arc  so  many.’  For  example,  there  are  over  1(X)  commercial 
graphical  user  interface  builders,  and  many  new  research  tools  ere  reported  every  year  at  conferences  like 
the  annual  ,\CM  User  Interface  Software  and  Technologv  "■  noosium  (L  IST)  and  the  .\CM  SIGCHI 
confererKe.  There  arc  also  about  three  PhD  theses  on  e  .•  acc  tools  every  year.  Therefore,  this 
article  provides  an  overview  of  the  most  popular  approaches,  ra  ..er  than  an  exhaustive  survey. 

2.  Definitions 

The  user  interface  (UI)  of  a  computer  program  is  the  pan  that  handles  the  output  to  die  displcv  and  the 
input  from  the  perwn  using  the  program.  The  rest  of  the  program  is  called  the  opplaati  r.  or  the 
application  semantics. 

User  interface  tools  have  been  called  various  names  over  the  years,  with  the  most  popular  being  i  scr 
Interface  Management  Systems  (UlMS)  [75  j.  However,  many  people  feel  that  the  term  UIMS  should  be 
used  only  for  tools  that  handle  the  sequencing  of  operations  (what  happens  after  each  event  from  the 
user),  so  other  terms  like  Toolkits,  User  Interface  Development  Environments.  Interface  Builders, 
Interface  Development  Tools,  and  Application  Frameworks  have  been  used.  This  paper  will  try  to  define 
these  terms  more  specifically,  and  use  the  general  term  "user  interface  tool”  for  all  software  aimed  to 
help  create  user  interfaces.  Note  that  the  word  "tool”  is  being  used  to  include  what  are  called 
"toolkits,”  as  well  as  higher-level  tools,  such  as  Interface  Builders,  that  are  not  toolkits. 

Four  different  classes  of  people  are  involved  with  user  interface  software,  and  it  is  important  to  have 
different  names  foi  them  to  avoid  confusion.  The  first  is  the  fjerson  using  the  resulting  program,  who  is 
called  the  end  user  or  just  user.  The  next  person  creates  the  user  interface  of  the  program,  and  is  called  the 
user  interface  designer  or  just  designer.  Working  with  the  user  interface  designer  will  be  the  person  who 
writes  the  software  for  the  rest  of  the  application.  This  person  is  called  the  application  programmer.  The 
designer  may  use  special  user  interface  tools  which  arc  provided  to  help  create  user  interfaces.  These 
tools  are  created  by  the  tool  creator.  Note  that  the  designer  will  be  a  u.scr  of  the  software  created  by  the 


partial  list  which  is  frequently  updated  is  available  through  Mosaic  or  other  world-widc-web  interfaces  as 
http : //WWW .cs . emu . edu ;8001/afs/cs. emu . edu/user/bam/www/toolnames . html 


User  Interface  Software  Tools 


-3 


tool  creator,  but  we  still  do  not  use  the  term  “user”  here  to  avoid  confusion  with  the  end  user.  Although 
this  classification  discusses  each  role  as  a  different  person,  in  fact,  there  may  be  many  people  in  each  role 
or  one  person  may  perform  multiple  roles.  The  general  term  programmer  is  used  for  anyone  who  writes 
code,  and  may  be  a  designer,  application  programmer,  or  tool  creator. 


3.  Importance  of  User  Interface  Tools 

There  are  many  advantages  to  using  user  interface  software  tools.  These  can  be  cla-ssified  into  two 
main  groups: 

•  The  quality  of  the  interfaces  will  be  higher.  This  is  because: 

•  E)esigns  can  be  rapidly  prototyped  and  implemented,  possibly  even  before  the 
application  code  is  written. 

•  It  is  easier  to  incorporate  changes  discovered  through  user  testing. 

•  There  can  be  multiple  user  interfaces  for  the  same  application. 

•  More  effort  can  be  expended  on  the  tool  than  may  be  practical  on  any  single  user 
interface  since  the  tool  wiU  be  used  with  many  different  applications. 

•  Different  applications  are  mote  likely  to  have  consistent  user  interfaces  if  they  arc 
created  using  the  same  user  interface  tool. 

•  It  will  be  easier  for  a  variety  of  specialists  to  be  involved  in  designing  the  user 
interface,  rather  than  having  the  user  interface  created  entirely  by  programmers. 
Graphic  artists,  cognitive  psychologists,  and  human  factors  specialists  may  all  be 
involved.  In  particular,  professional  user  interface  designers,  who  may  not  be 
programmers,  can  be  in  charge  of  the  overall  design. 

•  The  user  interface  code  will  be  easier  and  more  economical  to  create  and  maintain. 

This  is  because: 

•  Interface  specifications  can  be  represented,  validated,  and  evaluated  more  easily. 

•  There  will  be  less  code  to  write,  because  much  is  supplied  by  the  tools. 

■There  will  be  better  modularization  due  to  the  separation  of  the  user  interface 
component  from  the  application.  This  should  allow  the  user  interface  to  change 
without  affecting  the  application,  and  a  large  class  of  changes  to  the  application  (such 
as  changing  the  internal  algorithms)  should  be  possible  without  affecting  the  user 
interface. 

•  The  level  of  programming  expertise  of  the  interface  designers  and  implementors  can  be 
lower,  because  the  tools  hide  much  of  the  complexities  of  the  underlying  system. 

•  The  reliability  of  the  user  interface  will  be  higher,  since  the  code  for  the  u.scr  intcrfac'' 
is  created  automatically  from  a  higher  level  specification. 

•  It  will  be  easier  to  port  an  application  to  different  hardware  and  software  environments 
since  the  device  dependencies  are  isolated  in  the  user  interface  tool. 


Based  on  these  goals  for  user  interface  software  tools,  we  can  list  a  number  of  important  functions  that 
should  be  provided.  This  list  can  be  used  to  evaluate  the  various  tools  to  see  how  much  they  cover. 
Naturally,  no  tool  will  help  with  everything,  and  different  user  interface  designers  may  put  different 
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emphasis  on  the  different  features. 

In  general,  the  tools  might: 

•  help  design  the  interface  given  a  specification  of  the  end  users’  tasks, 

•  help  implement  the  interface  given  a  specification  of  the  design. 

•  help  evaluate  the  interface  after  it  is  designed  and  propose  improvements,  or  at  least  provide 
information  to  allow  the  designer  to  evaluate  the  interface, 

•  create  easy-to-use  interfaces, 

•  allow  the  Designer  to  rapidly  investigate  different  designs, 

•  allow  non-programmers  to  design  and  implement  user  interfaces, 

•  allow  the  end  user  to  customize  the  interface, 

•  provide  portability,  and 

•  be  easy  to  use  themselves. 

This  might  be  achieved  by  having  the  tools: 

•  automatically  choose  which  user  interface  styles,  input  devices,  widgets,  etc.  should  be  used, 

•  help  with  screen  layout  and  graphic  design, 

•  validate  user  inputs, 

•  handle  user  errors, 

•  handle  aborting  and  undoing  of  operations, 

•  provide  appropriate  feedback  to  show  that  inputs  have  been  received, 

•  provide  help  and  prompts, 

•  update  the  screen  display  when  application  data  changes. 

•  notify  the  application  when  the  user  updates  application  data. 

•  deal  with  field  scrolling  and  editing, 

•  help  with  the  sequencing  of  operations, 

•  insulate  the  application  from  all  device  dependencies  and  the  underlying  software  and 
hardware  systems, 

•  provide  customization  facilities  to  end  users,  and 

•  evaluate  the  graphic  design  and  layout,  usability,  aixl  learnability  of  the  interface. 


4.  Overview  of  User  Interface  Software  Tools 

Since  user  interface  software  is  so  difficult  to  create,  it  is  not  surprising  that  people  have  been  working 
for  a  long  time  to  create  tools  to  help  with  it  Today,  many  of  these  tools  and  ideas  have  progressed  from 
research  into  commercial  systems,  and  their  effectiveness  has  been  amply  demonstrated.  Research 
systems  also  continue  to  evolve  quickly,  and  the  models  that  were  popular  five  years  ago  have  been  made 
obsolete  by  more  effective  tools,  changes  in  the  computer  market  (c.g.,  the  demi.sc  of  OpenLook  will  take 
with  it  a  number  of  tools),  and  the  emergence  of  new  styles  of  u.scr  interfaces  such  as  pen-based 
computing  and  multi-media. 
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4.1  Components  of  User  Interface  Software 

As  shown  in  Figure  1,  user  interface  software  may  be  divided  into  various  layers:  the  windowing 
system,  the  toolkit  and  higher-level  tools.  Of  course,  many  practical  systems  span  multiple  layers. 


A  IP  IP  II  n  (S  a  ft  c®!!!! 

Higher-level  Tools 
Toolkit 

Windowing  System 

Operating  System 


Figure  1:  The  components  of  user  interface  software  discussed  in  this  article. 


The  windowing  system  supports  the  separation  of  the  screen  into  different  (usually  rcci;ingular)  regions, 
called  windows.  The  X  system  (911  divides  the  window  functionality  into  two  layers;  the  window  system. 
which  is  the  functional  or  programming  interface,  and  the  window  manager  which  is  the  user  interlace. 
Thus  the  “window  system”  provides  procedures  that  allow  the  application  to  draw  pictures  on  the  screen 
and  get  input  from  the  user,  and  the  “window  manager”  allows  the  end  user  to  move  windows  around, 
and  is  responsible  for  displaying  the  title  lines,  borders  and  icons  around  the  windows.  However,  many 
people  and  systems  use  the  name  “window  manager”  to  refer  to  both  layers,  since  systems  such  as  the 
Macintosh  and  Microsoft  Windows  do  not  separate  them.  This  article  will  use  the  X  terminology,  and  use 
the  term  “windowing  system”  when  referring  to  both  layers. 

On  top  of  the  windowing  system  is  the  toolkit,  which  contains  many  commonly  used  widgets  such  as 
menus,  buttons,  scroll  bars,  and  text  input  fields.  On  top  of  the  toolkit  might  be  higher-level  tools,  which 
help  the  designer  use  the  toolkit  widgets.  The  following  sections  discuss  each  of  these  components  in 
more  detail. 
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5.  Windowing  Systems 

A  windowing  system  is  a  software  package  that  helps  the  user  monitor  and  control  different  contexts  by 
separating  them  physically  onto  different  parts  of  one  or  more  display  screens.  A  survey  of  various 
windowing  systems  was  published  earlier  [47],  Although  most  of  today’s  systems  provide  toolkits  on  top 
of  the  windowing  systems,  as  wiU  be  explained  below,  toolkits  generally  only  address  the  drawing  of 
widgets  such  as  buttons,  menus  and  scroU  bars.  Thus,  when  the  programmer  wants  to  draw  application- 
specific  parts  of  the  interface  and  allow  the  user  to  manipulate  these,  the  window  system  interface  must  be 
used  directly.  Therefore,  the  windowing  system’s  programming  interface  has  significant  impact  on  most 
user  interface  programmers. 

The  first  windowing  systems  were  implemented  as  part  of  a  single  program  or  system.  For  example, 
the  EMACs  text  editor  {100],  and  the  Smalltalk  [112]  and  DLISP  [111]  programming  environments  had 
their  own  windowing  systems.  Later  systems  implemented  the  windowing  system  as  an  integral  part  of 
the  operating  system,  such  as  Sapphire  for  PERQs  [4S],  SunView  for  Suns  [106],  and  the  Macintosh, 
NeXT  and  Microsoft  Windows  systems.  In  order  to  allow  different  windowing  systems  to  operate  on  the 
same  operating  system,  some  windowing  systems,  such  as  X  and  Sun’s  NeWS,  operate  as  a  separate 
process,  and  use  the  operating  system’s  inter-process  communication  mechanism  to  connect  to 
applications. 

5.1  Structure  of  Windowing  Systems 

A  windowing  system  can  be  logically  divided  into  two  layers,  each  of  which  has  two  parts  (sec  Figure 
2).  The  window  system,  or  base  layer,  implements  the  basic  functionality  of  the  windowing  system.  The 
two  parts  of  this  layer  handle  the  display  of  graphics  in  windows  (the  output  model)  and  the  access  to  the 
various  input  devices  (the  input  model),  which  usually  includes  a  keyboard  and  a  pointing  device  such  as 
a  mouse.  The  primary  interface  of  the  base  layer  is  procedural,  and  is  called  the  windowing  system’s 
application  or  program  interface. 

The  other  layer  of  windowing  system  is  the  window  manager  or  user  interface.  This  includes  all 
aspects  that  arc  visible  to  the  user.  The  two  pans  of  the  user  interface  layer  arc  the  presentation,  which  is 
comprised  of  the  pictures  that  the  window  manager  displays,  and  the  corrunands,  which  arc  how  the  u.scr 
manipulates  the  windows  and  their  contents. 

5.2  Base  Layer 

The  base  layer  is  the  procedural  interface  to  the  windowing  system.  In  the  1970s  and  early  198()s,  there 
were  a  large  number  of  different  windowing  systems,  each  with  a  different  procedural  interface  (at  least 
one  for  each  hardware  platform).  People  writing  software  found  this  to  be  unacceptable  because  they 
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Figure  2:  The  windowing  system  can  be  divided  into  two  layers,  called  the  base  or  window  system 
layer,  and  the  user  interface  or  window  manager  layer.  Each  of  these  can  be  divided  into 
parts  th^^t  handle  output  and  input. 


wanted  to  be  able  to  run  tlveir  software  on  different  platforms,  but  they  would  have  to  rewrite  significant 
amounts  of  code  to  convert  from  one  window  system  to  another.  The  X  windowing  system  [91]  was 
created  to  solve  this  problem  by  providing  a  hardware-independent  interface  to  windows.  X  has  been 
quite  successfiil  at  this,  and  has  driven  virtually  all  other  witxiowing  systems  out  of  the  workstation 
hardware  market.  In  the  small  computer  market,  the  Macintosh  runs  its  own  window  system  or  X,  and 
IBM  PC-class  machines  primarily  run  Microsoft  Windows  or  IBM’s  Pre.scntation  Manager  (pan  of  OS/2). 

5.2.1  Output  Model 

The  output  model  is  the  set  of  procedures  that  an  application  can  u.se  to  draw  pictures  on  the  screen.  It 
is  important  that  all  output  be  directed  through  the  window  system  so  that  the  graphics  primitives  can  be 
clipped  to  the  window’s  borders.  For  exa..iple,  if  a  program  draws  a  line  that  would  extend  out  of  a 
window’s  borders,  it  must  be  clipped  so  that  the  contents  of  other,  independent,  windows  are  not 
overwritten.  Most  windowing  systems  provide  special  escapes  that  allow  programs  to  draw  directly  to  the 
screen,  without  using  the  window  system’s  clipping.  These  operations  can  be  much  quicker,  but  arc  very 
dangerous  and  therefore  should  seldom  be  used.  Most  modem  computers  provide  graphics  hardware  that 
is  specially  optimized  to  woilc  efficiently  with  the  window  system. 

In  early  windowing  systems,  such  as  Smalltalk  [1 12],  Blit  |85j  and  Sapphire  [461,  the  primary  output 
operation  was  BitBlt  (also  called  “RasierOp”).  These  systems  primarily  supported  monochrome  screens 
(each  pixel  is  either  black  or  white).  BitBlt  takes  a  rectangle  of  pixels  from  one  part  of  the  screen  and 
copies  it  to  another  part.  Various  boolean  operations  can  be  specified  for  combining  the  pixel  values  of 
the  source  and  destination  rectangles.  For  example,  the  source  rectangle  can  simply  replace  the 
destination,  or  it  might  be  XORed  with  the  destination.  BitBlt  can  be  used  to  draw  solid  rectangles  in 
either  black  or  white,  display  text,  scroll  windows,  aixl  perform  many  other  effects  [35].  The  only 
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additional  drawing  operation  typically  supported  by  these  early  systems  was  drawing  straight  lines. 

Later  windowing  systems,  such  as  the  Macintosh  and  X,  added  a  full  set  of  drawing  operations,  such  as 
filled  and  unfilled  polygons,  text,  lines,  arcs,  etc.  These  cannot  be  implemented  using  the  BitBlt  operator. 
With  the  growing  popularity  of  color  screens  and  non-rcctangular  primitives  (such  as  rounded  rectangles), 
the  use  of  BitBlt  has  significantly  decreased.  It  is  primarily  used  now  for  scrolling  and  copying  off-screen 
pictures  onto  the  screen  (e.g.,  to  implement  double-buffering). 

A  few  windowing  systems  allow  the  full  Postscript  imaging  model  [  1  ]  to  be  used  to  create  images  on 
the  screen.  Postscript  provides  device-independent  coordinate  systems  and  arbitrary  rotations  and  scaling 
for  all  objects,  including  text.  Another  advantage  of  using  Postscript  for  the  screen  is  that  the  same 
language  can  be  used  to  print  the  windows  on  paper  (since  many  printers  accept  Postscript).  Sun  created 
a  version  used  in  the  NeWS  windowing  system,  and  then  Adobe  (the  creator  of  Postscript)  came  out  with 
an  official  version  called  “Display  Postscript”  which  is  used  in  the  NeXT  windowing  system  and  is 
supplied  as  an  extension  to  the  X  windowing  system  by  a  number  of  vendors,  including  DEC  and  IBM. 

All  of  the  standard  ou^ut  models  only  contain  drawing  operations  for  two  dimensional  objects.  Two 
extensions  to  support  3-D  objects  ate  PEX  and  OpenGL.  PEX  [84]  is  an  extension  to  the  X  windowing 
system  that  incorporates  much  of  the  PHIGS  graphics  standard.  OpenGL  [78]  is  based  on  the  GL 
programming  interface  that  has  been  used  for  many  years  on  Silicon  Graphics  machines.  OpenGL 
provides  machine  independence  for  3-D  since  it  is  available  for  various  X  platforms  (SGI,  Sun,  etc.)  and 
is  included  as  a  standard  part  of  new  versions  of  Microsoft  Windows. 

As  shown  in  Figure  3,  the  earlier  windowing  systems  assumed  that  a  graphics  package  would  be 
implemented  using  the  windowing  system.  For  example,  the  CORE  graphics  package  was  implemented 
on  top  of  the  SunView  windowing  system.  All  newer  systems,  including  the  Macintosh,  X,  NeWS, 
NeXT,  and  Microsoft  Windows,  have  implemented  a  sophisticated  graphics  system  as  part  of  the 
windowing  system. 

5.2.2  Input  Model 

The  early  graphics  standards,  such  as  CORE  and  PHIGS,  provided  an  input  model  that  does  not  support 
the  modem,  direct  manipulation  style  of  interfaces.  In  those  standards,  the  programmer  calls  a  routine  to 
request  the  value  of  a  “virtual  device”  such  as  a  “locator”  (pointing  device  position),  “.string”  (edited 
text  string),  “choice”  (selection  from  a  menu),  or  “pick”  (selection  of  a  graphical  object).  The  program 
would  then  pause  waiting  for  the  user  to  take  action.  This  is  clearly  at  odds  with  the  direct  manipulation 
“mode-free”  style,  where  the  user  can  decide  whether  to  make  a  menu  choice,  select  an  object,  or  type 
something. 
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Figure  3:  Various  organizations  that  have  been  used  by  windowing  systems.  Boxes  with  exU'a  borders 
represent  systems  that  can  be  replaced  by  users.  Early  systems  (a)  tightly  coupled  the 
window  manager  and  the  window  system,  and  assumed  that  sophisticated  graphics  and 
toolkits  would  be  built  on  top.  The  next  step  in  designs  (b)  was  to  incorporate  into  the 
windowing  system  the  graphics  and  toolkits,  so  that  the  window  manager  itself  could  have  a 
more  sophisticated  look  and  feel,  and  so  applications  would  be  more  consistent.  Other 
systems  (c)  allow  different  window  managers  and  different  toolkits,  while  still  embedding 
sophisticated  graphics  packages. 
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With  the  advent  of  modem  windowing  systems,  a  new  model  was  provided;  a  stream  of  event  records 
is  sent  to  the  window  which  is  currently  accepting  input.  The  user  can  select  which  window  is  getting 
events  using  various  commands,  described  in  section  S.3.  Each  event  record  typically  contains  the  type 
and  value  of  the  event  (e.g.,  which  key  was  pressed),  the  window  that  the  event  was  directed  to,  a 
timestamp,  and  the  x  and  y  position  of  the  mouse.  The  windowing  system  queues  keyboard  events, 
mouse  button  events,  and  mouse  movement  events  together  (along  with  other  special  events)  and 
programs  must  dequeue  the  events  and  process  them.  It  is  somewhat  surprising  that,  although  there  has 
been  substantial  progress  in  the  output  model  for  windowing  systems  (from  BilBlt  to  complex  2-D 
primitives  to  3-D),  input  is  still  handled  in  essentially  this  same  way  today  as  in  the  original  windowing 
systems,  even  though  there  are  sf  e  well-known  unsolved  problems  with  this  model; 

•  There  is  no  provision  for  special  stop-output  (conlrol-S)  or  abort  (control-C,  command-dot) 
events,  so  these  will  be  queued  with  the  other  input  events. 

•  The  same  event  mechanism  is  used  to  pass  special  messages  from  the  windowing  system  to 
the  application.  When  a  window  gets  larger  or  becomes  uncovered,  the  application  must 
usually  be  notified  so  it  can  adjust  or  redraw  the  picture  in  the  window.  Most  window 
systems  communicate  this  by  enqueuing  special  events  into  the  the  event  stream,  which  the 
program  must  then  handle. 

•  The  application  must  always  be  willing  to  accept  events  in  order  to  process  aborts  and 
redrawing  requests.  If  not,  then  long  operations  cannot  be  aborted,  and  the  screen  may  have 
blank  areas  while  they  are  being  processed. 

•  The  model  is  device  dependent,  since  the  event  record  has  fixed  fields  for  the  expected 
incoming  events.  If  a  3-D  pointing  device  or  one  with  more  than  the  standard  number  of 
buttons  was  used  instead  of  a  mouse,  then  the  standard  event  mechanism  cannot  handle  it. 

•  Because  the  events  are  handled  asynchronously,  there  are  many  race  conditions  that  can 
cause  programs  to  get  out  of  synchronization  with  the  window  system.  For  example,  in  the  X 
windowing  system,  if  you  press  inside  a  window  and  release  outside,  under  certain  conditions 
the  program  will  think  that  the  mouse  button  is  still  depressed.  Another  example  is  that 
refr^h  requests  from  the  windowing  system  specify  a  rectangle  of  the  window  that  needs  to 
be  redrawn,  but  if  the  program  is  changing  the  contents  of  the  window,  the  wrong  area  may 
be  redrawn  by  the  time  the  event  is  processed.  This  problem  can  occur  when  the  window  is 
scrolled. 

Although  these  problems  have  been  known  for  a  long  time,  there  has  been  little  research  on  new  input 
models  (an  exception  is  the  Garnet  Interactors  model  [51]). 


5.2.3  Communication 

In  the  X  windowing  system  and  NeWS,  all  communication  between  applications  and  the  window 
system  uses  inter-process  communication  through  a  network  protocol.  This  means  that  the  application 
program  can  be  on  a  different  computer  from  its  windows.  In  all  other  windowing  systems,  operations 
are  implemented  by  dircc'Jy  calling  the  window  manager  procedures  or  through  special  traps  into  the 
operating  system.  The  primary  advantage  of  the  X  mechanism  is  that  it  makes  it  easier  for  a  person  to 
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.  j  multiple  machines  with  all  their  windows  appearing  on  a  single  machine.  Another  advantage  is 
'..ii  it  is  easier  to  provide  interfaces  for  different  programming  languages:  for  example  the  C  interface 
lied  xlib)  and  the  Lisp  interface  (called  CLX)  send  the  appropriate  messages  through  the  network 
-^oiocol.  The  primary  disadvantage  is  efficiency,  since  each  window  request  will  typically  be  encoded, 
'jsed  to  the  transport  layer,  and  then  decoded,  even  when  the  computation  and  windows  are  on  the  same 
hine. 


r  Interface  Layer 

interface  of  the  windowing  system  allows  the  user  to  control  the  windows.  In  X,  the  user  can 
:ch  user  interfaces,  by  killing  one  window  manager  and  starting  another.  Popular  window 
under  X  include  uwm  (which  has  no  title  lines  and  borders),  twm,  mwm  (the  Motif  window 
:r).  and  olwm  (the  OpenLook  window  manager).  There  is  a  standard  protocol  through  which 
irams  and  the  base  layer  communicate  to  the  window  manager,  so  that  aU  programs  continue  to  run 
•  ihout  change  when  the  window  manager  is  switched.  It  is  possible,  for  example,  to  run  applications 
It  use  Motif  widgets  inside  the  windows  controlled  by  the  OpenLook  window  manager. 

complete  discussion  of  the  options  for  the  user  interfaces  of  window  managers  was  previously 
isticd  [471.  Also,  the  video  All  the  Widgets  [52]  has  a  30  minute  segment  showing  many  different 
IS  t)t  window  manager  user  interfaces. 

me  parts  of  the  user  interface  of  a  windowing  system,  which  is  sometimes  called  its  “Irak  and  feel,” 
pparcntly  be  copyrighted  and  patented.  Which  parts  is  a  highly  complex  is.suc,  and  the  status 
.js  with  decisions  in  various  court  cases.  Good  references  for  more  information  are  the  “Legally 
•ising ' '  columns  of  Communications  of  the  ACM  [90]. 

.1  Presentation 

i  he  presentation  of  the  windows  defines  how  the  screen  looks.  One  very  important  aspect  of  the 
cscniaiion  of  windows  is  whether  they  can  overlap  or  not.  Overlapping  windows,  sometimes  called 
'•red  windows,  allow  one  window  to  be  partially  or  totally  on  top  of  another  window,  as  shown  in 
'  This  is  also  sometimes  called  the  desktop  metaphor,  since  windows  can  cover  each  other  like 
;  paper  can  cover  each  other  on  a  desk.^  The  other  alternative  is  called  tiled  windows,  which 
ii  windows  are  not  allowed  to  cover  each  other.  Figure  5  shows  an  example  of  tiled  windows, 
a  window  manager  that  supports  covered  windows  can  also  allow  them  to  be  side-by-side,  but 
.  versa.  Therefore,  a  window  manager  is  classified  as  “covered”  if  it  allows  windows  to  overlap. 


;  cre  .iru  usually  Other  aspects  to  the  desktop  metaphor,  however,  such  as  presenting  file  operations  in  a  way  that  mimics 
:>fx:raiions,  as  in  the  Star  office  workstation  [98]. 
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The  tiled  style  was  popular  for  awhile,  and  was  used  by  Cedar  [1091,  and  early  versions  of  the  Star  [98], 

Andrew  [82],  and  Microsoft  Windows.  A  study  even  suggested  that  using  tiled  windows  was  more 
efficient  for  users  [6].  However,  today  tiled  windows  are  rarely  seen,  because  users  generally  prefer 
overlapping. 
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Figure  4:  A  screen  from  the  Macintosh  showing  3  windows  covering  each  other,  and  some  icons  along 
the  right  margin. 


Another  important  aspect  of  the  presentation  of  windows  is  the  use  of  icons  (also  shown  in  Figures  4 
and  5).  These  are  small  pictures  that  represent  windows  (or  sometimes  files).  They  are  used  because 
there  would  otherwise  be  too  many  windows  to  conveniently  fit  on  the  screen  and  manage.  Other  aspects 
of  the  presentation  include  whether  the  window  has  a  title  line  or  not,  what  the  background  (where  there 
are  no  windows)  looks  like,  and  whether  the  title  and  borders  have  control  areas  for  performing  window 
operations. 
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Figures:  A  screen  from  the  Cedar  [109]  windowing  system.  Windows  are  “tiled”  into  2  columns. 

There  is  a  row  of  icoas  along  the  bottom.  Each  window  has  a  fixed  menu  of  commands 
below  the  lillc  line. 


5.3.2  Commands 

Since  computers  typically  have  multiple  windows  and  only  one  mouse  and  keyboard,  there  must  be  a 
way  for  the  user  to  control  which  window’  is  getting  keyboard  input.  This  window  is  called  the  input  for 
keyboard)  focus.  .•Another  term  is  the  listener  since  it  is  listening  to  the  user’s  typing.  Some  systems 
called  the  focus  the  “active  window"  or  "current  window,”  but  these  are  poor  terms  since  in  a  multi¬ 
processing  system,  many  windows  can  be  actively  outputting  information  at  the  same  lime.  Window 
managers  provide  vanous  ways  to  specify  and  show  which  window  is  the  listener.  The  most  imponant 
options  arc; 

•  click-to-type.  which  means  that  the  u,scr  mu.st  click  the  mouse  button  in  a  window  before 
typing  to  It.  This  is  u.scd  by  the  .Macintosh. 

•  muve-to-type,  which  means  that  the  .mouse  only  has  to  move  over  a  window  to  allow  typing 
to  it.  This  is  usually  faster  for  the  user,  but  may  cause  input  to  go  to  the  wrong  window  if  the 
user  accidentally  knocks  the  mouse. 

Most  X  window  managers  (including  the  Motif  and  OpenLook  window  managers)  allow  the  user  to 
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choose  which  method  is  desired.  However,  the  choice  can  have  significant  impact  on  ihc  user  imcrlace  ot 
applications.  For  example,  because  the  Macintosh  requires  click-lo-typc.  it  can  provide  a  single  menu  bar 
at  the  top,  and  the  commands  can  always  operate  on  the  focussed  window.  With  move-io-type.  the  user 
might  have  to  pass  through  various  windows  (thus  giving  them  the  focus)  on  the  way  to  the  top  of  the 
screen.  Therefore,  Motif  applications  must  have  a  menubar  in  each  window  so  the  commands  will  know 
which  window  to  operate  on. 

All  covered  window  systems  allow  the  user  to  change  which  window  is  on  top  (not  covered  by  other 
windows),  and  usually  to  send  a  window  to  the  bottom  (covered  by  all  other  windows).  Other  commands 
allow  windows  to  be  changed  size,  moved,  created  and  destroyed. 

6.  Toolkits 

A  toolkit  is  a  library  of  “widgets”  that  can  be  called  by  application  programs.  A  widget  is  a  way  of 
using  a  physical  input  device  to  input  a  certain  type  of  value.  Typically,  widgets  in  toolkits  include 
menus,  buttons,  scroll  bars,  text  type-in  fields,  etc.  Figure  6  shows  some  examples  of  widgets.  Creating 
an  interface  using  a  toolkit  can  only  be  done  by  programmers,  because  toolkits  only  have  a  procedural 
interface. 

Using  a  toolkit  has  the  advantage  that  the  final  UI  will  look  and  act  similarly  to  other  UIs  created  using 
the  same  toolkit,  and  each  application  does  not  have  to  re-write  the  standard  functions,  such  as  menus.  A 
problem  with  toolkits  is  that  the  styles  of  interaction  are  limited  to  those  provided.  For  example,  it  is 
difficult  to  create  a  single  slider  that  contains  two  indicators,  which  might  be  useful  to  input  the  upper  and 
lower  bounds  of  a  range.  In  addition,  the  toolkits  themselves  are  often  expensive  to  create:  “The 
primitives  never  seem  complex  in  pritKiple,  but  the  programs  that  implement  them  are  surprisingly 
intricate”  (13,  p.l99].  Another  problem  with  toolkits  is  that  they  are  often  difficult  to  use  since  they  may 
contain  hundreds  of  procedures,  and  it  is  often  not  clear  how  to  use  the  procedures  to  create  a  desired 
interface.  For  example,  the  documentation  for  the  Macintosh  Toolbox  now  covers  six  books,  of  which 
about  1/3  is  related  to  user  interface  programming. 

As  with  the  graphics  package,  the  toolkit  can  be  implemented  either  using  or  being  used  by  the 
windowing  system  (see  Figure  3).  Early  systems  provided  only  minimal  widgets  (e.g.,  just  a  menu),  and 
expected  applications  to  provide  others.  In  the  Macintosh,  the  toolkit  is  at  a  low  level,  and  the  window 
manager  user  interface  is  built  using  it.  The  advantage  of  this  is  that  the  window  manager  can  then  use 
the  same  sophisticated  toolkit  routines  for  its  user  interface.  When  the  X  system  was  being  developed, 
the  developers  could  not  agree  on  a  single  toolkit,  so  they  left  the  toolkit  to  be  on  top  of  the  windowing 
system.  In  X,  programmers  can  use  a  variety  of  toolkits  (for  example,  the  xt[44],  Interviews  (42], 
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Figure  6:  Some  of  the  widgets  with  a  Motif  look -and-fcel  provided  by  the  Garnet  toolkit. 


Garnet  [53]  or  tk  [80]  toolkits  can  be  used  on  top  of  X),  but  the  window  manager  must  usually  implement 
its  user  interface  from  scratch. 

Because  the  designers  of  X  could  not  agree  on  a  single  look-and-feel,  they  created  an  intrinsics  layer  on 
which  to  build  different  widget  sets,  which  they  called  xt  [44],  This  layer  provides  the  common  services, 
such  as  techniques  for  object-oriented  programming  and  layout  control.  The  widget  set  layer  is  the 
collection  of  widgets  that  is  implemented  using  the  intrinsics.  Multiple  widget  sets  with  different  looks 
and  feels  can  be  implemented  on  top  of  the  same  intrinsics  layer  (Figure  7-a),  or  else  the  same  look-and- 
feel  can  be  implemented  on  top  of  different  intrinsics  (Figure  7-b).  Recently,  Sun  announced  that  it  was 
phasing  out  OpenLx)ok,  which  means  that  X  and  xi  will  be  standardized  on  the  Motif  widget  set. 
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(a)  (b) 

Figure?:  (a)  At  least  three  different  widget  sets  that  have  different  looks  and  feels  have  been 
implemented  on  top  of  the  xt  intrinsics.  (b)  The  Motif  look-and-feci  has  been  implemented 
on  at  least  three  different  intrinsics. 


6.1  Toolkit  Intrinsics 

Toolkits  come  in  two  basic  varieties.  The  most  conventional  is  simply  a  collection  of  procedures  that 
can  be  called  by  application  programs.  Examples  of  this  style  include  the  SunTools  toolkit  for  the 
SunView  windowing  system  (106),  and  the  Macinto.sh  Toolbox  12).  The  ou.cr  variety  uses  an  object- 
oriented  programming  style  which  makes  it  easier  for  the  designer  to  customize  the  interaction 
techniques.  Examples  include  Smalltalk  11 12|,  Andrew  (821Gamct  153),  Interviews  (42|.  and  Xt  [44|. 

The  advantages  of  using  object-oriented  intrinsics  arc  that  it  is  a  natural  way  to  think  about  widgets  (the 
menus  and  buttons  on  the  screen  seem  like  objects),  the  widget  objects  can  handle  some  of  the  chores  that 
otherwise  would  be  left  to  the  programmer  (such  as  refresh),  and  it  is  easier  to  create  custom  widgets  (by 
sub-classing  an  existing  widget).  The  advantage  of  the  older,  procedural  style  is  that  it  is  easier  to 
implement,  no  special  object-oriented  system  is  needed,  and  it  is  easier  to  interface  to  multiple 
programming  languages. 

To  implement  the  objects,  the  toolkit  might  invent  its  own  object  system,  as  was  done  with  Xt.  Andrew 
and  Garnet,  or  it  might  use  an  existing  object  system,  as  was  done  in  Interviews  [42]  which  uses  C++, 
NeXTStep  [68]  which  uses  Objective-C,  and  Rendezvous  [30]  which  uses  CLOS  (the  standard  Common 
Lisp  Object  System). 

The  usual  way  that  object-oriented  toolkits  interface  with  application  programs  is  through  the  u.sc  of 
call-back  procedures.  These  are  procedures  defined  by  the  application  programmer  that  arc  called  when  a 
widget  is  operated  by  the  end  user.  For  example,  the  programmer  might  supply  a  procedure  to  be  called 
when  the  user  selects  a  menu  item.  Experience  has  shown  that  real  interfaces  often  contain  hundreds  of 
call-backs,  which  makes  the  code  harder  to  modify  and  maintain  [57].  In  addition,  different  toolkits,  even 
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when  implemented  on  the  same  intrinsics  like  Motif  and  OpenLook,  have  different  call-back  protocols. 
This  means  that  code  for  one  toolkit  is  difficult  to  port  to  a  different  toolkit.  Therefore,  research  is  being 
directed  at  reducing  the  number  of  call-backs  in  user  interface  software  [5S]. 

Some  research  toolkits  have  added  novel  things  to  the  toolkit  intrinsics.  For  example.  Garnet  [531, 
Rendezvous  [311,  and  Bramble  [211  allow  the  objects  to  be  connected  using  constraints,  which  arc 
relationships  that  are  declared  once  and  then  maintained  automatically  by  the  system.  For  example,  the 
designer  can  specify  that  the  color  of  a  rectangle  is  constrained  to  be  the  value  of  a  slider,  and  then  the 
system  will  automatically  update  the  rectangle  if  the  user  moves  the  slider. 

6.2  Widget  Set 

Typically,  the  intrinsics  layer  is  look-and-feel  independent,  which  means  that  the  widgets  built  on  top 
of  it  can  have  any  desired  appearance  and  behavior.  However,  a  particular  widget  set  must  pick  a  look- 
and-feel.  The  video  All  the  Widgets  shows  many  examples  of  widgets  that  have  been  dcsigiKd  over  the 
years  [521.  For  example,  it  shows  35  different  kinds  of  menus.  Like  window  manager  user  interfaces,  the 
widgets’  look-and-feel  can  be  copyrighted  and  patented  (90). 

As  was  mentioned  above,  different  widget  sets  (with  different  looks  and  feels)  can  be  implemented  on 
lop  of  the  same  intrinsics.  In  addition,  the  same  look-and-feel  can  be  implemented  on  top  of  different 
intrinsics.  For  example,  there  are  Motif  look-and-fccl  widgets  on  top  of  the  xi.  Interviews  and  Garnet 
intrinsics  (Figure  7-b).  Although  they  all  look  and  operate  the  same  (.so  would  be  indistinguishable  to  the 
end  user),  they  are  implemented  quite  differently,  and  have  completely  different  procedural  interfaces  for 
the  programmer. 

6.3  Specialized  Toolkits 

A  number  of  toolkits  have  been  developed  to  support  specific  kinds  of  applications  or  specific  classes 
of  programmers.  For  example,  the  SUIT  system  [83]  (which  contains  a  toolkit  and  an  interface  builder), 
is  specifically  designed  to  be  easy  to  learn  and  is  aimed  at  classroom  instruction.  Garnet  [53]  provides 
high-level  support  for  graphical,  direct  manipulation  interfaces,  and  includes  a  toolkit,  interface  builder 
and  other  high-level  tools.  Rendezvous  [30]  is  designed  to  make  it  easier  to  create  applications  that 
support  multiple  users  on  multiple  machines  operating  synchronously.  Whereas  most  toolkits  only 
provide  2-D  interaction  techniques,  the  Brown  Animation  Generation  System  [128]  and  Silicon  Graphics’ 
Inventor  toolkit  [102, 121]  provide  preprogrammed  3-D  widgets  and  a  framework  for  creating  others. 
The  Ttoolkit  [23]  provides  built-in  primitives  for  controlling  the  timing  of  an  interface,  which  is 
important  for  supporting  multi-media,  such  as  video.  Special  support  for  animations  has  been  added  to 
Artkit,  including  motion  blur,  timing  and  curved  trajectories  [33]. 
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Tk  [80]  is  a  popular  toolkit  for  the  X  window  system  because  it  uses  an  interpretive  language  called  tcl 
which  makes  it  possible  to  dynamically  change  the  user  interface.  Tcl  also  supports  the  Unix  style  o( 
programming  where  many  small  programs  are  glued  together. 

7.  Virtual  Toolkits 

Although  there  are  many  small  differences  among  the  various  toolkits,  much  remains  the  same.  For 
example,  all  have  some  type  of  menu,  button,  scroll  bar,  text  input  field,  etc.  Although  there  are  fewer 
windowing  systems  and  toolkits  than  there  were  five  years  ago.  people  are  still  finding  that  they  must  do  a 
lot  of  work  to  convert  their  software  from  Motif  to  OpenLook  to  the  Macintosh  and  to  Microsoft 
Windows. 

Therefore,  a  number  of  systems  have  been  developed  that  try  to  hide  the  differences  among  the  various 
toolkits,  by  providing  virtual  widgets  which  can  be  mapped  into  the  widgets  of  each  toolkit.  Another 
name  for  these  tools  is  cross-platform  development  systems.  The  programmer  writes  the  code  once  using 
the  virtual  toolkit  and  the  code  will  run  without  change  on  different  platforms  and  still  look  like  it  was 
designed  with  that  platform’s  widgets.  For  example,  the  virtual  toolkit  might  provide  a  .single  menu 
routine,  which  always  has  the  same  programmer  interface,  but  connects  to  a  Motif  menu,  Macintosh  menu 
or  a  Windows  menu  depending  on  which  machitK  the  application  is  run  on.  A  recent  report 
[15]  compares  a  number  of  virtual  toolkits. 

There  are  two  styles  of  virtual  toolkits.  In  one,  the  virtual  toolkit  links  to  the  different  actual  toolkits  on 
the  host  machine.  For  example,  XVT  [127]  provides  a  C  or  C-h-  interface  that  links  to  the  actual  Motif. 
OpenLook,  Macintosh,  MS-Windows,  and  OS/2-PM  toolkits  (and  also  character  terminals)  and  hides 
their  differences.  The  second  style  of  virtual  toolkit  re-implements  the  widgets  in  each  style.  For 
example.  Galaxy  [116]  and  Open  Interface  [77]  provide  libraries  of  widgets  that  look  like  those  on  the 
various  platforms.  The  advantage  of  the  first  style  is  that  the  user  interface  is  more  likely  to  be  look-and- 
feel  conformant  (since  it  uses  the  real  widgets).  The  disadvantages  are  that  the  virtual  toolkit  must  still 
provide  an  interface  to  the  graphical  drawing  primitives  on  the  platfoims.  Furthermore,  they  tend  to  only 
provide  functions  that  appear  in  all  toolkits.  Many  of  the  virtual  toolkits  that  take  the  .second  approach, 
for  example  Galaxy,  provide  a  sophisticated  graphics  package  and  complete  sets  of  widgets  on  all 
platforms.  However,  with  the  second  approach,  there  must  always  be  a  large  run-time  library,  since  in 
addition  to  the  built-in  widgets  that  are  native  to  the  machine,  there  is  the  rc-implcmcntation  of  these 
same  widgets  in  the  virtual  toolkit  library. 


All  of  the  toolkits  that  work  on  multiple  platforms  can  be  considered  virtual  toolkits  of  the  second  type. 
For  example,  SUIT  [83]  works  on  X,  Macintosh  and  Windows,  and  Garnet  [53]  works  on  X  and  the 
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Macintosh.  However,  these  use  the  same  iook-and-feel  on  ail  platforms  (and  therefore  do  not  look  the 
same  as  the  other  applications  on  that  platform),  so  they  are  not  classified  as  virtual  toolkits. 


8.  Higher  Level  Tools 

Since  programming  at  the  toolkit  level  is  quite  difficult,  there  is  a  tremendous  interest  in  higher  level 
tools  that  will  make  the  user  interface  software  production  process  easier.  These  are  discussed  next. 

8.1  Phases 

Many  higher-level  tools  have  components  that  operate  at  different  times.  The  design  time  component 
helps  the  user  interface  designer  design  the  user  interface.  For  example,  this  might  be  a  graphical  editor 
which  can  lay  out  the  interface,  or  a  compiler  to  process  a  user  interface  specification  language.  The  next 
phase  is  when  the  end-user  is  using  the  program.  Here,  the  run-time  component  of  the  tool  is  used.  This 
usually  includes  a  toolkit,  but  may  also  include  additional  software  specifically  for  the  tool.  Since  the 
run-time  component  is  '‘managing”  the  user  interface,  the  term  User  Interface  Management  System 
seems  appropriate  for  tools  with  a  significant  run-time  component. 

There  may  also  be  an  qfter-run-time  component  that  helps  with  the  evaluation  and  debugging  of  the 
user  interface.  Unfortunately,  vei^  few  user  interface  tools  have  an  after-run-time  component.  This  is 
partially  because  tools  that  have  tried,  such  as  MIKE  (73],  discovered  that  there  are  very  few  metrics  that 
can  be  applied  by  computers.  A  new  generation  of  tools  are  trying  to  evaluate  how  people  will  interact 
with  interfaces  by  automatically  creating  cognitive  models  from  high-level  descriptions  of  the  user 
interface.  For  example,  USAGE  creates  an  NGOMSL  cognitive  model  from  a  UIDE  user  interface 
specification  [12]. 

8.2  Specification  Styles 

High-level  user  interface  tools  come  in  a  large  variety  of  forms.  One  important  way  that  they  can  be 
classified  is  by  how  the  designer  specifies  what  the  interface  should  be.  As  shown  in  Figure  8,  some  tools 
require  the  programmer  to  program  in  a  special-purpose  language,  some  provide  an  application 
framework  to  guide  the  programming,  some  automatically  generate  the  interface  from  a  high-level  model 
or  specification,  and  others  allow  the  interface  to  be  designed  interactively.  Each  of  these  types  is 
discussed  below.  Of  course,  some  tools  use  different  techniques  for  specifying  different  parts  of  the  user 
interface.  These  are  classified  by  their  predominant  or  most  interesting  feature. 
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Figure  8:  Ways  to  specify  the  user  interface,  some  tools  that  use  that  technique,  and  the  section  of  this 
article  that  discusses  the  technique. 
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8J.1  Language  Based 

With  most  of  the  older  user  interface  tools,  the  designer  specified  the  user  interface  in  a  special-purpose 
language.  This  language  can  take  many  fomis,  including  context-free  grammars,  state  transition 
diagrams,  declarative  languages,  event  languages,  etc.  The  language  is  usually  used  to  specify  the  syntax 
of  the  user  interface;  i.e.,  the  legal  sequences  of  input  and  output  actions.  This  is  someiimes  called  the 
"dialog.”  Green  [22]  provides  an  extensive  comparison  of  grammars,  state  transition  diagrams,  and 
event  languages,  and  Olsen  [75]  surveys  various  DIMS  techniques. 

8J.1.1  State  Transition  Networks 

Since  many  parts  of  user  interfaces  involve  handling  a  sequence  of  input  events,  it  is  natural  to  think  of 
using  a  state  transition  network  to  code  the  interface.  A  transition  network  consists  of  a  set  of  states,  with 
arcs  out  of  each  state  labeled  with  the  input  tokens  that  will  cause  a  transition  to  the  state  at  the  other  end 
of  the  arc.  In  addition  to  input  tokens,  calls  to  application  procedures  and  the  output  to  display  can  also 
be  put  on  the  arcs  in  some  systems.  Newman  implemented  a  simple  tool  using  finite  state  machines  in 
1968  [67]  which  handled  textual  input.  This  was  apparently  the  first  user  interface  tool.  Many  of  the 
assumptions  and  techniques  used  in  modem  systems  were  present  in  Newman's;  different  languages  for 
defining  the  user  interface  and  the  semantics  (the  semantic  routines  were  coded  in  a  normal  programming 
language),  a  table-driven  syntax  analyzer,  and  device  independence. 

State  diagram  tools  are  most  useful  for  creating  user  interfaces  where  the  user  interface  has  a  large 
number  of  modes  (each  state  is  really  a  mode).  For  example,  state  diagrams  are  useful  for  describing  the 
operation  of  low-level  widgets  (e.g.,  how  a  menu  or  scroll  bar  works),  or  the  overall  global  flow  of  an 
application  (e.g.,  this  command  will  pop-up  a  dialog  box.  from  which  you  can  get  to  these  two  dialog 
boxes,  and  then  to  this  other  window,  etc.).  However,  most  highly-interactive  systems  attempt  to  be 
mostly  '  ‘mode-hee,’’  which  means  that  at  each  point,  tlK  user  has  a  wide  variety  of  choices  of  what  to  do. 
This  requires  a  large  number  of  arcs  out  of  each  state,  so  state  diagram  tools  have  not  been  successful  for 
these  interfaces.  In  addition,  state  diagrams  cannot  handle  interfaces  where  the  user  can  operate  on 
multiple  objects  at  the  same  time.  Another  problem  is  that  they  can  be  very  confusing  for  large  interfaces, 
sirKC  they  get  to  be  a  “maze  of  wires”  and  off-page  (or  off-screen)  arcs  can  be  hard  to  follow. 

Recognizing  these  problems,  but  still  trying  to  retain  the  perspicuousness  of  state  transition  diagrams, 
Jacob  [36]  invented  a  new  formalism,  which  is  a  combination  of  state  diagrams  with  a  form  of  event 
languages  (see  section  8.2. 1.3).  There  can  be  multiple  diagrams  active  at  the  same  time,  and  flow  of 
control  transfers  from  one  to  another  in  a  co-routine  fashion.  The  system  can  create  various  forms  of 
direct  manipulation  interfaces.  VAPS  is  a  commercial  system  that  uses  the  state  transition  model,  and  it 
eliminates  the  maze-of-wires  problem  by  providing  a  spreadsheet-like  table  in  which  the  states,  events, 
and  actions  are  specified  [115].  Transition  networks  have  been  thoroughly  researched,  but  have  not 
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proven  particularly  successful  or  useful  either  as  a  research  or  conunercial  approach. 

8JL1 J!  Context-Free  Grammars 

Many  grammar-based  systems  are  based  on  parser  generators  used  in  compiler  development.  For 
example,  the  designer  might  specify  the  user  interface  syntax  using  some  form  of  BNF.  Examples  of 
grammar-based  systems  are  Syngraph  [70]  and  parsers  built  with  YACC  and  LEX  in  Unix. 

Grammar-based  tools,  like  state  diagram  tools,  arc  not  appropriate  for  specifying  highly-intcractive 
interfaces,  since  they  arc  oriented  to  batch  processing  of  strings  with  a  complex  syntactic  structure.  These 
systems  arc  best  for  textual  command  languages,  and  have  been  mostly  abandoned  for  user  interfaces  by 
researchers  and  commercial  developers. 

8J.U  Event  Languages 

With  event  languages,  the  input  tokens  arc  considered  to  be  “events”  that  are  sent  to  individual  event 
handlers.  Each  handler  will  have  a  condition  clause  that  determines  what  types  of  events  it  will  handle, 
and  when  it  is  active.  The  body  of  the  handler  can  cause  output  events,  change  the  internal  state  of  the 
system  (which  might  enable  other  event  handlers),  or  call  application  routines. 

The  ALGAE  system  [19]  uses  an  event  language  which  is  an  extension  of  Pa.scal.  The  user  interface  is 
programmed  as  a  set  of  small  event  handlers,  which  ALGAE  compiles  into  conventional  code.  Sassafras 
[29],  uses  a  similar  idea,  but  with  an  entirely  different  syntax.  Sassafras  also  adds  local  variables  called 
"flags”  to  help  specify  the  flow  of  control.  As  described  below  in  section  8.2.4.2,  the  HypeiTalk 
language  that  is  part  of  HyperCard  for  the  Apple  Macintosh,  can  also  be  considered  an  event  language. 

The  advantages  of  event  languages  are  that  they  can  handle  multiple  input  devices  active  at  the  same 
time,  and  it  is  straightforward  to  support  non-modal  interfaces,  where  the  user  can  operate  on  any  widget 
or  object  The  main  disadvantage  is  that  it  can  be  very  difficult  to  create  correct  code,  since  the  flow  of 
control  is  not  localized  and  small  changes  in  one  part  can  affect  many  different  pieces  of  the  program.  It 
is  also  typically  difficult  for  the  designer  to  urxlerstand  the  code  once  it  reaches  a  non-trivial  size. 
However,  the  success  of  HyperTalk  and  similar  tools  shows  that  this  approach  is  appropriate  for  small  to 
medium-size  programs. 

8.2.1.4  Declarative  Languages 

Another  approach  is  to  try  to  define  a  language  that  is  declarative  (stating  what  should  happen)  rather 
than  procedural  (how  to  make  it  happen).  Cousin  [27]  and  HP/ApoUo’s  Open-Dialogue  [93]  both  allow 
the  designer  to  specify  user  interfaces  in  this  manner.  The  user  interfaces  supported  arc  basically  forms, 
where  fields  can  be  text  which  is  typed  by  the  user,  or  options  selected  using  menus  or  buttons.  There  are 
also  graphic  output  areas  that  the  application  can  use  in  whatever  manner  desired.  The  application 
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program  is  connected  to  the  user  interface  tfirough  “variables”  which  can  be  set  and  accessed  by  both. 
As  researchers  have  extertded  this  idea  to  support  more  sophisticated  interactions,  the  specification  has 
grown  into  full  application  models,  and  newer  systems  are  described  in  section  8.2.3. 

Another  type  of  declarative  language  is  the  layout  description  languages  that  come  with  many  'oolkits. 
For  example.  Motifs  User  Interface  Language  (UIL)  allows  the  layout  of  widgets  to  be  defined.  Since 
the  UIL  is  interpreted  when  an  application  starts,  users  can  (in  theory)  edit  the  UIL  code  to  customize  the 
interface.  UIL  is  not  a  complete  language,  however,  in  the  sense  that  the  designer  must  still  write  C  code 
for  many  parts  of  the  interface,  including  any  areas  containing  dynamic  graphics  and  any  widgets  that 
change. 

The  advantage  of  using  declarative  languages  is  that  the  user  interface  designer  does  not  have  to  worry 
about  the  time  sequence  of  events,  and  can  concentrate  on  the  information  that  needs  to  be  passed  back 
and  forth.  The  disadvantage  is  that  only  certain  types  of  interfaces. can  be  provided  this  way,  and  the  rest 
must  be  programmed  by  hand  in  the  “graphic  areas”  provided  to  application  programs.  The  kinds  of 
interactions  available  are  preprogrammed  and  fixed.  In  particular,  these  systems  provide  no  support  for 
such  things  as  dragging  graphical  objects,  rubber-band  lines,  drawing  new  graphical  objects,  or  even 
dynamically  changing  the  items  in  a  menu  based  on  the  application  mode  or  context.  However,  these 
languages  are  now  proving  successful  as  intermediate  languages  describing  the  layout  of  widgets  (such  as 
UIL)  that  ate  generated  by  interactive  tools.  They  were  also  an  important  intermediate  research  step  on 
the  way  to  today’s  model-based  approaches  (section  8.2.3). 

8,2.1,5  Constraint  Languages 

A  number  of  user  interface  tools  allow  the  programmer  to  use  constraints  to  define  the  user  interface 
[9].  Early  constraint  systems  include  Sketchpad  [108]  which  pioneered  the  use  of  graphical  constraints 
in  a  drawing  editor,  and  Thinglab  [8]  which  used  constraints  for  graphical  simulation.  Subsequently, 
Thinglab  was  extended  to  aid  in  the  generation  of  user  interfaces  [9]. 

Section  6.1  mentioned  the  use  of  constraints  as  part  of  the  intrinsics  of  a  toolkit  A  number  of  research 
toolkits  now  supply  constraints  as  an  integral  part  of  the  object  system  (e.g..  Garnet  [53]).  In  addition, 
some  systems  have  provided  higher-level  interfaces  to  constraints.  Graphical  Thinglab  [10]  allows  the 
designer  to  create  constraints  by  wiring  icons  together,  and  NoPump[1241,  C32  [56]  and  Penguims 
[34]  allow  constraints  to  be  defined  using  spreadsheet-like  interfaces. 

The  advantages  of  constraints  is  that  they  ate  a  natural  way  to  express  many  kinds  of  relationships  that 
arise  frequently  in  user  interfaces.  For  example,  that  lines  should  stay  attached  to  boxes,  that  labels 
should  stay  centered  within  boxes,  etc.  However,  a  disadvantage  with  constraints  is  that  they  require  a 
soi^ticated  run-time  system  to  solve  them  efficiently.  Another  problem  is  that  they  can  be  hard  to 
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debug  when  specified  incorrectly  since  it  can  be  difficult  to  trace  the  causes  and  consequences  of  values 
changing.  However,  a  growing  number  of  research  systems  are  using  constraints,  and  it  aj^ars  that 
modem  constraint  solvers  and  debugging  techniques  may  solve  these  problems,  so  constraints  have  a 
great  potential  to  simplify  the  programming  task.  As  yet.  there  are  no  commercial  user  interface  tools 
using  constraints. 

S2.1.6  Screen  Scrapers 

Some  commercial  tools  are  specialized  to  be  “front-enders"  or  “screen  .scrapers”  which  provide  a 
graphical  user  interface  to  old  programs  without  changing  the  existing  application  code.  They  do  this  by 
providing  an  in-memory  buffer  that  pretends  to  be  the  screen  of  an  old  character  terminal  such  as  might 
be  attached  to  an  IBM  mainframe.  When  the  mainframe  application  outputs  to  the  buffer,  a  program  the 
designer  writes  in  a  special  programming  language  converts  this  into  an  update  of  a  graphical  widget. 
Similarly,  when  the  user  operates  a  widget,  the  script  converts  this  into  the  appropriate  edits  of  the 
character  buffer.  The  leading  program  of  this  type  is  Basel  [17],  which  tiIso  contains  an  interface  builder 
for  laying  out  the  widgets. 

8J.1.7  Database  Interfaces 

A  very  important  class  of  commercial  tools  support  form-based  or  GUl-bascd  access  to  databases. 
Major  database  vendors  such  as  Oracle  [79]  provide  tools  which  allow  designers  to  define  the  user 
interface  for  accessing  and  setting  data.  Often  these  tools  include  interactive  forni  editors  (which  are 
essentially  interface  builders)  and  special  database  languages.  Fourth  generation  languages  (4GLs),  that 
support  defining  the  interactive  forms  for  accessing  and  entering  data,  also  fall  into  this  category. 

8J.1,8  Visual  Programming 

“Visual  programs”  use  graphics  and  two  (or  more)  dimensional  layout  as  part  of  the  program 
specification  [54].  Many  different  approaches  to  using  visual  programming  to  specify  user  interfaces 
have  been  investigated.  Most  systems  thav  support  state  transition  networks  (section  8.2. 1. 1)  use  a  visual 
representation.  Another  popular  technique  is  to  use  dataflow  languages.  In  these,  icons  represent 
processing  steps,  and  the  data  flow  along  the  connecting  wires.  The  user  interface  is  usually  constructed 
directly  by  laying  out  pre-built  widgets,  in  the  style  of  interface  builders  (section  8.2.4.3).  Examples  of 
visual  programming  systems  for  creating  user  interfaces  include  Labview  [40]  which  is  specialized  for 
controlling  laboratory  instruments,  and  ProGraph  [87]  Using  a  visual  language  seems  to  make  it  easier 
for  novice  programmers,  but  large  programs  still  suffer  from  the  familiar  “maze  of  wires”  problem. 
Other  papers  (e.g.,  [54])  have  analyzed  the  strengths  and  weaknesses  of  visual  programming  in  detail. 

Another  popular  language  is  Visual  Basic  from  Microsoft  [117].  Although  this  is  more  of  a  structure 
editor  for  Basic  comlrined  with  an  interface  builder,  and  therefore  does  not  really  count  as  a  visual 
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language,  it  does  make  the  construction  of  user  interface  software  easier.  Microsoft  is  pushing  Visual 
Basic  as  the  extension  language  that  people  will  use  to  customize  and  connect  all  future  Windows-based 
applications. 

8J1.1.9  Summary  of  Language  Approaches 

In  summary,  there  have  been  many  different  types  of  languages  that  have  been  designed  for  specifying 
user  interfaces.  One  problem  with  all  of  these  is  that  they  can  only  be  used  by  professional  programmers. 
Some  programmers  have  objected  to  the  requirement  for  learning  a  new  language  for  programming  Just 
the  user  interface  portion  [72].  This  has  been  confinned  by  market  research  [126,  p.29].  Furthermore,  it 
seems  more  natural  to  define  the  graphical  part  of  a  user  interface  using  a  graphical  editor  (see  section 
8.2.4).  However,  it  is  clear  that  for  the  foreseeable  future,  much  of  the  user  interface,  win  still  need  to  be 
created  by  writing  programs,  so  it  is  appropriate  to  continue  investigations  into  the  best  language  to  use 
for  this.  Indeed  a  new  book  is  entirely  devoted  to  investigating  the  languages  for  programming  user 
interfaces  [58]. 

8.2.2  Application  Frameworks 

After  the  Macintosh  Toolbox  had  been  available  for  a  little  while,  Apple  discovered  that  programmers 
had  a  difficult  time  figuring  out  how  to  call  the  various  toolkit  functions,  and  how  to  ensure  that  the 
resulting  interface  met  the  Apple  guidelines.  They  therefore  created  a  software  system  that  provides  an 
overall  application  framework  to  guide  programmers.  This  is  called  MacApp[92, 125]  and  uses  the 
object-oriented  language  Object  Pascal.  Gasses  are  provided  for  the  important  parts  of  an  application, 
such  as  the  main  windows,  the  commands,  etc.,  and  the  programmer  specializes  these  classes  to  provide 
the  application-specific  details,  such  as  what  is  actually  drawn  in  the  windows  and  which  commands  arc 
provided.  MacApp  has  been  very  successful  at  simplifying  the  writing  of  Macintosh  applications. 

Unidraw  [119]  uses  a  similar  approach,  but  it  is  more  specialized  for  graphical  editors.  This  means  that 
it  can  provide  even  more  support.  Unidraw  uses  the  C-h-  object-oriented  language  and  is  part  of  the 
Interviews  system  [42].  Unidraw  has  been  used  to  create  various  drawing  and  CAD  programs,  and  a  user 
interface  editor  [120].  The  Garnet  framework  is  also  aimed  at  graphical  applications,  but  due  to  its 
graphical  data  model,  many  of  the  built-in  routines  can  be  used  without  change  (the  programmer  docs  not 
usually  need  to  write  methods  for  subcla.sses)  [59].  The  ACE  system  from  HP  provides  an  interactive 
editor  that  allows  some  of  the  properties  of  objects  to  be  specified,  but  most  of  the  application-specific 
behavior  must  still  be  programmed  [37].  Even  more  specialized  are  various  graph  programs,  such  as 
Edge  [66]  and  TGE  [38].  These  provide  a  framework  in  which  the  designer  can  create  programs  that 
display  their  data  as  trees  or  graphs.  The  programmer  typically  specializes  the  node  and  arc  classes,  and 
specifies  some  of  the  commands,  but  the  framework  handles  layout  and  the  overall  control. 
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An  emerging  popular  approach  aims  to  replace  today’s  large,  monolythic  applications  with  smaller 
components  that  attach  together.  For  example,  you  might  buy  a  separate  text  editor,  ruler,  paragraph 
foimatter,  spell  checker,  and  drawing  program,  and  have  them  all  work  together  seamlessly.  This 
tq^roach  was  invented  by  the  Andrew  environment  [82]  which  provides  an  object-oriented  document 
model  that  supports  the  embedding  of  different  kinds  of  data  inside  other  documents.  These  “insets’  ’  are 
unlike  data  that  is  cut  and  pasted  in  systems  like  the  Macintosh  because  they  bring  along  the  programs 
that  edit  them,  and  therefore  can  always  be  edited  in  place.  Furthermore,  the  container  document  does  not 
need  to  know  how  to  display  or  print  the  inset  data  since  the  original  program  that  created  it  is  always 
available.  The  designer  creating  a  new  inset  writes  subclasses  that  adheres  to  a  standard  protocol  so  the 
system  knows  how  to  pass  input  events  to  the  appropriate  editor.  The  next  generation  of  operating 
systems  will  use  this  approach  extensively:  it  is  the  foundation  for  Microsoft’s  OLE  and  Apple’s 
OpenDoc. 

All  of  these  frameworks  require  the  designer  to  write  code,  typically  by  creating  application-specific 
sub-classes  of  the  standard  classes  provided  as  part  of  the  framework. 

Another  class  of  systems  that  might  be  considered  “frameworks”  help  create  user  interfaces  that  arc 
composed  of  a  series  of  “cards,”  such  as  HyperCard  from  Apple.  These  systems  are  discussed  in  section 
8.2.4.2  because  their  primary  interface  to  the  designer  is  graphical. 

8.2.3  Model-Based  Automatic  Generation 

A  problem  with  all  of  the  language-based  tools  is  that  the  designer  must  specify  a  great  deal  about  the 
placement,  format,  and  design  of  the  user  interfaces.  To  solve  this  problem,  some  tools  use  automatic 
generation  so  that  the  tool  makes  many  of  these  choices  from  a  much  higher-level  specification.  Many  of 
these  tools,  including  Mickey  [74],  Jade  [114],  Chisel  [95],  and  DON  [39]  have  concentrated  on  creating 
menus  and  dialog  boxes.  Chisel  and  Jade  allow  the  designer  to  use  a  graphical  editor  to  edit  the  generated 
interface  if  it  is  not  good  enough.  DON  has  the  most  sophisticated  layout  mechanisms  and  takes  into 
account  the  desired  window  size,  balance,  columness,  symmetry,  grouping,  etc.  Creating  dialog  boxes 
automatically  has  been  very  thoroughly  researched,  but  there  still  are  no  commercial  tools  that  do  this. 

Another  approach  is  to  try  to  create  a  user  interface  based  on  a  list  of  the  application  procedures. 
MIKE  [71]  creates  an  initial  interface  that  is  menu-oriented  and  rather  verbose,  but  the  designer  can 
change  the  menu  structure,  use  icons  for  some  commands,  and  even  make  some  commands  operate  by 
direct  manipulation.  The  designer  uses  a  graphical  editor,  like  those  described  in  section  8.2.4,  to  specify 
these  changes. 

UIDE  (the  User-Interface  Design  Environment)  [104]  requires  that  the  semantics  of  the  application  be 
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defined  in  a  special-purpose  language,  and  therefore  might  be  included  with  the  language-based  tools 
(section  8.2.1).  It  is  placed  here  instead  because  the  language  is  used  to  describe  the  functions  that  the 
{plication  supports  and  not  the  desired  interface.  UIDE  is  classified  as  a  “model-based"  approach 
because  the  specification  serves  as  a  high-level,  sophisticated  model  of  the  application  semantics.  In 
UIDE,  the  description  includes  pre-  and  post-conditions  of  the  operations,  and  the  system  uses  these  to 
reason  about  the  operations,  and  to  automatically  generate  an  interface.  One  interesting  part  of  this 
system  is  that  the  user  interface  designer  can  apply  “transformations”  to  the  interface.  These  change  the 
interface  in  various  ways.  For  example,  one  transformation  changes  the  interface  to  have  a  currently 
selected  object  instead  of  requiring  an  object  to  be  selected  for  each  operation.  UIDE  applies  the 
transformations  and  insures  that  the  resulting  interface  remains  consistent  Another  feature  of  UIDE  is 
that  the  pre-  and  post-conditions  are  used  to  automatically  generate  help  [103].  One  direction  of  current 
research  is  to  make  UIDE  models  easier  to  create  by  allowing  users  to  demonstrate  some  parts  of  the 
interface  [20].- 

Another  model-based  system  is  HUMANOID  [110]  which  supports  the  modeling  of  the  presentation, 
behavior  and  dialogue  of  an  interface.  The  HUMANOID  modeling  language  includes  abstraction, 
composition,  recursion,  iteration  and  conditional  constructs  to  support  sophisticated  interfaces.  The 
HUMANOID  system,  which  is  built  on  top  of  the  Garnet  toolkit  [53],  provides  a  number  of  interactive 
modeling  tools  to  help  the  designer  specify  the  model.  The  developers  of  HUMANOID  and  UIDE  are 
collaborating  on  a  new  combined  model  called  MASTERMIND  that  integrates  their  approaches  [65]. 

The  ITS  [123]  system  also  uses  rules  to  generate  an  interface.  ITS  was  used  to  create  the  visitor 
information  system  for  the  EXPO  1992  worlds  fair  in  Seville,  Spain.  Unlike  the  other  rule-based  systems, 
the  designer  using  ITS  is  expected  to  write  many  of  the  rules,  rather  than  just  writing  a  specification  that 
the  rules  work  on.  In  particular,  the  design  philosophy  of  ITS  is  that  all  design  decisions  should  be 
codified  as  rules  so  that  they  can  be  used  by  subsequent  designers,  which  will  hopefully  mean  that 
interface  designs  will  get  easier  and  better  as  more  rules  are  entered.  As  a  result,  the  designer  should 
never  use  graphical  editing  to  improve  the  design,  since  then  the  system  cannot  capture  the  reason  that  the 
generated  design  was  not  sufficient. 

While  the  idea  of  having  the  user  interface  generated  automatically  is  appealing,  this  approach  is  still  at 
the  research  level,  because  the  user  interfaces  that  arc  generated  arc  generally  not  good  enough.  A  further 
problem  is  that  the  specification  languages  can  be  quite  hard  to  learn  and  use.  Extensive  current  research 
is  addressing  the  problems  of  expanding  the  range  of  what  can  be  created  automatically  (to  go  beyond 
dialog  boxes)  and  to  make  the  model-based  approach  easier  to  use. 
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8J.4  Direct  Graphical  Specification 

The  tools  described  next  all  allow  the  user  interface  to  be  defined,  at  least  partially,  by  placing  objects 
on  the  screen  using  a  pointing  device.  This  is  motivated  by  the  observation  that  the  visual  presentation  of 
the  user  interface  is  of  primary  importance  in  graphical  user  interfaces,  and  a  graphical  tool  seems  to  be 
the  most  appropriate  way  to  specify  the  graphical  appearance.  Another  advantage  of  this  technique  is  that 
it  is  usually  much  easier  for  the  designer  to  use.  Many  of  these  systems  can  be  used  by  non-programmers. 
Therefore,  psychologists,  graphic  designers  and  user  interface  specialists  can  more  easily  be  involved  in 
the  user  interface  design  process  when  these  tools  are  used. 

These  tools  can  be  distinguished  from  those  that  use  “visual  programming’’  (section  8.2. 1.8)  since  with 
direct'  graphical  specification,  the  actual  user  interface  (or  a  part  of  it)  is  drawn,  rather  being  generated 
indirectly  by  a  visual  program.  Thus,  direct  gr^hical  specification  tools  have  been  called  direct 
manipulation  programming  since  the  user  is  directly  manipulating  the  user  interface  widgets  and  other 
elements. 

The  tools  that  support  graphical  specification  can  be  classified  into  four  categories;  prototyping  tools, 
those  that  support  a  sequence  of  cards,  interface  builders,  and  editors  for  application-specific  graphics. 

8 J.4.1  Prototyping  Tools 

The  goal  of  prototyping  tools  is  to  allow  the  designer  to  quickly  mock  up  some  examples  of  what  the 
screens  in  the  program  will  look  like.  Often,  these  tools  cannot  be  used  to  create  the  real  user  interface  of 
the  program;  they  just  show  how  some  aspects  will  look.  This  is  the  chief  factor  that  distinguishes  them 
from  other  high-level  tools.  Many  parts  of  the  interface  may  not  be  operable,  and  some  of  the  things  that 
look  like  widgets  may  just  be  static  pictures.  In  most  prototypers,  no  real  toolkit  widgets  are  used,  which 
means  that  the  designer  has  to  draw  simulations  that  look  like  the  widgets  that  will  appear  in  the  interface. 
The  normal  use  is  that  the  designer  would  spend  a  few  days  or  weeks  trying  out  different  designs  with  the 
tool,  and  then  completely  reimplement  the  final  design  in  a  separate  system.  Most  prototyping  tools  can 
be  used  without  programming,  so  they  can.  for  example,  be  used  by  graphic  designers. 

Note  that  this  use  of  the  term  “prototyping”  is  different  from  the  general  phrase  “rapid  prototyping,” 
which  has  become  a  maiketing  buzz-word.  Advertisements  for  just  about  all  user  interface  tools  claim 
that  they  support  “rapid  prototyping,”  by  which  they  mean  that  the  tool  helps  create  the  user  interface 
software  quicker.  The  term  “prototyping”  is  being  used  in  this  paper  in  a  much  more  specific  manner. 

Probably  the  first  prototyping  tool  was  Dan  Bricklin’s  Demo  [89].  This  is  a  program  for  an  IBM  PC 
that  allows  t’lc  designer  to  create  sample  screens  composed  of  characters  and  “character  graphics” 
(where  the  fixed-size  character  cells  can  contain  a  graphic  like  a  horizontal,  vertical  or  diagonal  line). 
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The  designer  can  easily  create  the  various  screens  for  the  application.  It  is  also  relatively  easy  to  specify 
the  actions  (mouse  or  keyboard)  that  cause  transitions  from  one  screen  to  another.  However,  it  is  difficult 
to  define  other  behaviors.  In  general,  there  may  be  some  support  for  type-in  fields  and  menus  in 
prototyping  tools,  but  there  is  little  ability  to  process  or  test  the  results. 

For  graphical  user  interfaces,  designers  often  use  tools  like  Director  [43]  for  the  Macintosh  which  is 
actually  an  animation  tool.  The  designer  can  draw  example  screens,  and  then  specify  that  when  the 
mouse  is  pressed  in  a  particular  place,  an  animation  should  start  or  a  different  screen  should  be  displayed. 
Components  of  the  picture  can  be  reused  in  different  screens,  but  again  the  ability  to  show  behavior  is 
limited.  HyperCard  for  the  Macintosh  is  also  often  used  as  a  prototyping  tool. 

The  primary  disadvantage  of  these  prototyping  tools  is  that  they  cannot  create  the  actual  code  for  the 
user  interface.  Therefore,  the  interfaces  must  be  re-coded  after  prototyping.  There  is  also  the  risk  that  the 
progratiuners  who  implement  the  teal  user  interface  will  ignore  the  prototype.  Therefore,  a  new  research 
tool  is  trying  to  provide  a  quick  sketching  interface  and  then  convert  the  sketches  into  actual  widgets  [41]. 

8J.4J  Cards 

Many  graphical  programs  are  limited  to  user  interfaces  that  can  be  presented  as  a  sequence  of  mostly 
static  pages,  sometimes  called  “frames,”  “cards,”  or  “forms.”  Each  page  contains  a  set  of  widgets, 
some  of  which  cause  transfer  to  other  pages.  There  is  usually  a  fixed  set  of  widgets  to  choose  from, 
which  were  coded  by  hand. 

An  early  example  of  this  is  Menulay[ll],  which  allows  the  designer  to  place  text,  graphical 
potentiometers,  iconic  pictures,  and  light  buttons  on  the  screen  and  see  exactly  what  the  end  user  will  see 
when  the  application  is  run.  The  designer  does  not  need  to  be  a  programmer  to  use  Menulay.  Trillium 
[28],  which  is  aimed  at  designing  the  user  interface  panels  for  photocopiers,  is  very  similar  to  Menulay. 
One  strong  advantage  that  Trillium  has  over  Menulay  is  that  the  cards  can  be  executed  immediately  as 
they  are  designed  since  the  specification  is  interpreted  rather  than  compiled.  Trillium  also  separates  the 
behavior  of  interactions  from  the  graphic  presentation  and  allows  the  designer  to  change  the  graphics 
(while  keeping  the  same  behavior)  without  programming.  One  weakness  is  that  it  has  little  support  for 
ffame-to-frame  transitions,  since  this  rarely  is  necessary  for  j^otocopiers. 

Probably,  the  most  famous  example  of  a  card-based  system  is  HyperCard  from  Apple.  There  arc  now 
many  similar  programs,  such  as  GUIDE  [81],  Spinnaker  Plus  [99],  and  Tool  Book  [3].  In  all  of  these,  the 
designer  can  easily  create  cards  containing  text  fields,  buttons,  etc.,  along  with  various  graphic 
decorations.  The  buttons  can  transfer  to  other  cards.  These  programs  provide  a  scripting  language  to 
provide  more  flexibility  for  buttons.  HyperCard’s  scripting  language  is  called  HypeiTalk,  and  as 
mentioned  in  section  8.2. 1.3,  is  really  an  event  language,  since  the  programmer  writes  short  pieces  of 
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code  that  are  executed  when  input  events  occur. 

8J.4J  Interface  Builders 

An  interface  builder  allows  the  designer  to  create  dialog  boxes,  menus  and  windows  that  are  to  be  part 
of  a  larger  user  interface.  These  are  also  called  Interface  Development  Tools.  Interface  builders  allow  the 
designer  to  select  from  a  pre-defined  library  of  widgets,  and  place  them  on  the  screen  using  a  mouse. 
Other  properties  of  the  widgets  can  be  set  using  property  sheets.  Usually,  there  is  also  some  support  for 
sequencing,  such  as  bringing  up  sub-dialogs  when  a  particular  button  is  hit  The  Steamer  project  at  BBN 
demonstrated  many  of  the  ideas  later  incorporated  into  interface  builders  and  was  probably  the  first 
object-oriented  graphics  system  [101].  Other  examples  of  research  interface  builders  are  DialogEditoi 
[14],  vu  [94]  and  Gilt  [55].  There  are  literally  hundreds  of  commercial  interface  builders.  Just  a  few 
examples  are  the  NeXT  Interface  Builder  [68],  Prototyper  for  the  Macintosh  [97],  WindowsMAKER  for 
Microsoft  Windows  on  the  PC  [5],  UIMX  for  X  Windows  and  Motif  [118],  and  devGuide  from  Sun  for 
OpenLook  [107].  Many  of  the  tools  discussed  above,  such  as  the  virtual  toolkits,  visual  languages,  and 
application  frameworks,  also  contain  interface  builders. 

Interface  builders  use  the  actual  widgets  from  a  toolkit,  so  they  can  be  used  to  build  parts  of  real 
applications.  Most  will  generate  C  code  templates  that  can  be  compiled  along  with  the  application  code. 
Others  generate  a  description  of  the  interface  in  a  language  that  can  be  read  at  run-time.  For  example. 
UIMX  generates  a  UIL  description.  It  is  usually  important  that  the  programmer  not  edit  the  output  of  the 
tools  (such  as  the  generated  C  code)  or  else  the  tool  can  no  longer  be  used  for  later  modifications. 

Although  interface  builders  make  laying  out  the  dialog  boxes  and  menus  easier,  this  is  only  part  of  the 
user  interface  design  problem.  These  tools  provide  little  guidance  towards  creating  good  user  interfaces, 
siiKe  they  give  designers  significant  freedom.  Another  problem  is  that  for  any  kind  of  program  that  has  a 
graphics  area  (such  as  drawing  programs,  CAD,  visual  language  editors,  etc.),  interface  builders  do  not 
help  with  the  contents  of  the  graphics  pane.  Also,  they  cannot  handle  widgets  that  change  dynamically. 
For  example  if  the  contents  of  a  menu  or  the  layout  of  a  dialog  box  changes  based  on  program  state,  this 
must  be  programmed  by  writing  code.  To  help  with  this  part  of  the  problem,  some  interface  builders,  like 
UIMX  [118],  provide  a  C  code  inteqireter. 

8J.4.4  Data  Visualization  Tools 

An  important  commercial  category  of  tools  are  dynamic  data  visualization  systems.  Thc.se  tools,  which 
tend  to  be  quite  expensive,  emphasize  the  display  of  dynamically  changing  data  on  a  computer,  and  are 
used  as  front  ends  for  simulations,  process  control,  system  monitoring,  network  management,  and  data 
analysis.  The  interface  to  the  designer  is  usually  quite  similar  to  an  interface  builder,  with  a  palette  of 
gauges,  graphers,  knobs  and  switches  that  can  be  placed  interactively.  However,  these  controls  usually 
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are  not  from  a  toolkit  and  are  supplied  by  the  tool.  Example  tools  in  this  category  include  DataViews 
[113],  SL-GMS  [96],  and  VAPS  [115]. 

8^.43  Editors  for  Application-Specific  Graphics 

When  an  application  has  custom  graphics,  it  would  be  useful  if  the  designer  could  draw  pictures  of 
what  the  graphics  should  look  like  rather  than  having  to  write  code  for  this.  The  problem  is  that  the 
graphic  objects  usually  need  to  change  at  run  time,  based  on  the  actual  data  and  end  user’s  actions. 
Therefore,  the  designer  can  only  draw  an  example  of  the  desired  display,  which  will  be  modified  at 
run-time,  and  so  these  tools  are  called  “demonstrational  programming”  [60].  This  distinguishes  these 
programs  from  the  graphical  tools  of  the  previous  three  sections,  where  the  full  picture  can  be  specified  at 
design  time.  As  a  result  of  the  generalization  task  of  convening  the  example  objects  into  parameterized 
prototypes  that  can  change  at  nm-time,  most  of  these  systems  are  still  in  the  research  phase. 

Peridot  [48]  allows  new,  custom  widgets  to  be  created.  The  primitives  that  the  designer  manipulates 
with  the  mouse  are  rectangles,  circles,  text,  and  lines.  The  system  generalizes  from  the  designer’s  actions 
to  create  parameterized,  object-oriented  procedures  like  those  that  might  be  found  in  toolkits. 
Experiments  showed  that  Peridot  can  be  used  by  non-programmers.  Lapidary  [50]  extends  the  ideas  of 
Peridot  to  allow  general  application-specific  objects  to  be  drawn.  For  example,  the  designer  can  draw  the 
nodes  and  arcs  for  a  graph  program.  The  DEMO  system  [18]  allows  some  dynamic,  run-time  properties 
of  the  objects  to  be  demonstrated,  such  as  how  objects  are  created.  The  Marquise  tool  [631  allows  the 
designer  to  demonstrate  when  various  behaviors  should  happen,  and  supports  palettes  which  control  the 
behaviors.  Research  continues  on  making  these  ideas  practical. 

8.3  Specialized  Tools 

For  some  application  domains,  there  are  customized  tools  that  provide  significant  high-level  support. 
These  tend  to  be  quite  expensive,  however  (i.e.,  US$20,(XX)  to  US$50,000).  For  example,  in  the 
aeronautics  and  real-time  control  areas,  there  are  a  number  of  high-level  tools,  including  Autocode 
[4]  and  InterMAPhics  [86]. 

9.  Technology  Transfer 

User  interface  tools  are  an  area  where  research  has  had  a  tremendous  impact  on  the  current  practice  of 
software  development.  Of  course,  window  managers  and  the  resulting  ‘‘GUI  style”  comes  from  the 
seminal  research  at  the  Stanford  Research  Institute,  Xerox  Palo  Alto  Research  Center,  and  MIT  in  the 
1970s.  Interface  builders  and  “card”  programs  like  HyperCard  were  invented  in  research  labs  at  BBN, 
the  University  of  Toronto,  Xerox  PARC,  and  others.  Now,  interface  builders  are  at  least  a  US$100 
million  per  year  business  and  are  widely  used  for  commercial  software  development  Event  languages,  as 
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widely  used  in  HyperTalk  and  elsewhere,  were  first  investigated  in  research  labs.  The  next  generation  of 
environments,  like  OLE  and  OpenDoc,  will  be  based  on  the  component  architecture  which  was  developed 
in  the  Andrew  environment  from  CMU.  Thus,  whereas  some  early  UlMS  approaches  like  transition 
networks  and  grammars  may  not  have  been  successful,  overall,  the  user  interface  tool  research  has 
changed  the  way  that  software  is  developed. 


10.  Evaluating  User  Interface  Tools 

There  are  clearly  a  large  number  of  approaches  to  how  tools  work,  and  there  are  research  and 
commercial  tools  that  use  each  of  the  techniques.  When  faced  with  a  particular  programming  task,  the 
designer  might  ask  which  tool  is  the  most  appropriate.  Different  approaches  are  appropriate  for  different 
kinds  of  tasks,  and  orthogonally,  there  are  some  dimensions  that  are  useful  for  evaluating  all  tools.  An 
important  point  is  that  in  today’s  market,  there  is  probably  a  commercial  higher-level  tool  appropriate  for 
most  tasks,  so  if  you  are  programming  directly  at  the  window  manager  or  even  toolkit  layer,  there  may  be 
a  tool  that  will  save  you  much  work. 

10.1  Approaches 

Using  the  commercial  tools,  if  you  are  designing  a  command-line  style  interface,  then  a  parser- 
generator  like  YACC  and  Lex  is  appropriate.  If  you  are  creating  a  graphical  application,  then  you  should 
definitely  be  using  a  toolkit  appropriate  to  your  platform.  If  there  is  an  application  framework  available, 
it  will  probably  be  very  helpful.  For  creating  the  dialog  boxes  and  menus,  an  interface  builder  is  very 
useful,  and  generally  easier  to  use  than  declarative  languages  like  UIL.  If  your  application  is  entirely  (or 
mostly)  pages  of  information  with  some  fields  for  the  user  to  fill  in,  then  the  card  tools  might  be 
t^ropriate. 

Among  the  approaches  that  are  still  in  the  research  phase,  constraints  seem  quite  appropriate  for 
specifying  graphical  relationships,  automatic  generation  may  be  useful  for  dialog  boxes  and  menus,  and 
graphical  editors  will  allow  the  graphical  elements  of  the  user  interface  to  be  drawn. 

There  is  a  big  debate  going  on  about  the  model-based  and  direct  graphical  specification  approaches 
[122, 105).  The  model-based  tools  provide  a  top-down  (or  “application-out”)  approach  where  the 
functions  are  specified  first,  whereas  the  graphical  tools  provide  a  bottom-up  (or  “user-interface -in”) 
approach  where  the  user  interface  is  designed  before  the  functions.  Furthermore,  the  automatic,  model- 
based  approaches  seem  to  provide  too  little  flexibility  to  the  designer,  whereas  the  graf^cal  tools  provide 
too  much  flexibility  and  not  enough  guidance.  Some  researchers  are  trying  to  create  systems  that 
combine  both  approaches  to  try  to  achieve  the  advantages  of  both  [20], 
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10.2  Dimensions 

There  are  many  dimensions  along  which  you  might  evaluate  user  interface  tools.  The  importance  given 
to  these  different  factors  will  depend  on  the  type  of  application  to  be  created,  and  the  needs  of  the 
designers. 

•  Depth.  How  much  of  the  user  interface  does  the  tool  cover?  For  example,  Interface  Builders 
help  with  dialog  boxes,  but  do  not  help  with  creating  interactive  graphics.  Does  the  tool  help 
with  the  evaluation  of  the  interfaces? 

•  Breadth.  How  many  different  user  interface  styles  are  supported,  or  is  the  resulting  user 
interface  limited  to  just  one  style,  such  as  a  sequence  of  cards?  If  this  is  a  higher-level  tool, 
does  it  cover  all  the  widgets  in  the  underlying  toolkit?  Can  new  interaction  techniques  and 
widgets  be  added  if  necessary? 

•  Portability.  Will  the  resulting  user  interface  run  on  multiple  platforms,  such  as  X, 
Macintosh  and  Windows? 

•  Ease  of  use  of  tools.  How  difficult  are  the  tools  to  use?  For  toolkits  and  most  language- 
based  higher-level  tools,  highly-trained  professional  programmers  are  needed.  For  some 
graphical  tools,  even  inexperienced  end-users  can  generate  user  interfaces.  Also,  since  the 
designers  are  themselves  users  of  the  tools,  the  conventional  user-interface  principles  can  be 
used  to  evaluate  the  quality  of  the  tools’  own  user  interface. 

•  Efficiency  for  designers.  How  fast  can  designers  create  user  interfaces  with  the  tool?  This 
is  often  related  to  the  quality  of  the  user  interface  of  the  tool. 

•  Quality  of  resulting  interfaces.  Does  the  tool  generate  high-quality  user  interfaces?  Docs 
the  tool  help  the  designer  evaluate  and  improve  the  quality?  Many  tools  allow  the  designer 
to  produce  any  interface  desired,  so  they  provide  no  specific  help  in  improving  the  quality  of 
the  user  interfaces. 

•  Performance  of  resulting  interface.  How  fast  does  the  resulting  user  interface  operate? 

Some  tools  interpret  the  specifications  at  run-time,  or  provide  many  layers  of  software,  which 
may  make  the  resulting  user  interface  too  slow  on  some  target  machines.  Another 
consideration  is  the  space  overhead  since  some  tools  require  large  libraries  to  be  in  memory 
at  run-time. 

•  Price.  Some  tools  are  provided  free  by  research  organizations,  such  as  the  xt  toolkit  from 
MIT  and  Garnet  from  C^U.  Most  personal  computers  and  workstations  today  come  with  a 
free  toolkit.  Commercial  higher  level  tools  can  range  from  $200  to  $50,000,  depending  on 
their  capabilities. 

•  Robustness  and  Support.  In  one  study,  users  of  many  of  the  commercial  tools  complained 
about  bugs  even  in  the  officially  released  version  [57],  so  checking  for  robustness  is 
important.  Since  many  of  the  tools  are  quite  hard  to  use,  the  level  of  training  and  support 
provided  by  the  vendor  might  be  important. 


Naturally,  there  are  tradeoffs  among  these  criteria.  Generally,  tools  that  have  the  most  power  (depth 
and  breadth)  are  more  difficult  to  use.  The  tools  that  arc  easiest  to  use  might  be  most  efficient  for  the 
designer,  but  not  if  they  cannot  create  the  desired  interfaces. 


As  tools  become  more  widespread,  reviews  and  evaluations  of  them  are  beginning  to  appear  in 
magazines  such  as  Open  Systems  Today  for  Unix  and  PC  Magazine.  Market  research  firms  are  writing 
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reports  evaluating  various  tools  [126, 24, 16].  Also,  there  are  a  few  formal  studies  of  tools  [32], 

11.  Research  Issues 

Although  there  are  many  user  interface  tools,  there  are  plenty  of  areas  in  which  further  research  is 
needed.  A  report  prepared  for  an  NSF  study  discusses  future  research  ideas  for  user  interface  tools  at 
length  [76].  Here,  a  few  of  the  important  ones  are  summarized. 

11.1  New  Programming  Languages 

The  built-in  input/output  primitives  in  today’s  programming  languages  support  a  textual  question- and- 
answer  style  of  user  interface  which  is  modal  and  well-known  to  be  poor.  Most  of- today’s  tools  use 
libraries  and  interactive  programs  which  are  separate  from  programming  languages.  However,  many  of 
the  techniques,  such  as  object-oriented  programming,  multiple-processing,  and  constraints, .  are  best 
provided  as  part  of  the  programming  language.  Furthermore,  an  integrated  environment,  where  the 
graphical  parts  of  an  application  can  be  specified  graphically  and  the  rest  textually,  would  make  the 
generation  of  a^)plications  much  easier.  A  new  book  discusses  how  programming  languages  can  be 
improved  to  better  support  user  interface  software  [58]. 

11.2  Increased  Depth 

Many  researchers  are  trying  to  create  tools  that  will  cover  more  of  the  user  interface,  such  as 
application-specific  graphics  and  behaviors.  The  challenge  here  is  to  allow  flexibility  to  application 
developers  while  still  providing  a  high  level  of  support.  Tools  should  also  be  able  to  support  Help,  Undo, 
and  Aborting  of  operations. 

Today’s  user  interface  tools  mostly  help  with  the  generation  of  the  code  of  the  interface,  and  assume 
that  the  fundamental  user  interface  design  is  complete.  What  are  also  needed  are  tools  to  help  with  the 
generation,  specification,  and  analysis  of  the  design  of  the  interface  [41].  For  example,  an  important  first 
step  in  user  interface  design  is  task  analysis,  where  the  designer  identifies  the  particular  tasks  that  the  end 
user  will  need  to  perform.  Research  should  be  directed  at  creating  tools  to  support  these  methods  and 
techniques.  These  might  eventually  be  integrated  with  the  code  generation  tools,  so  that  the  information 
generated  during  eariy  design  can  be  fed  into  automatic  generation  tools,  possibly  to  produce  an  interface 
directly  from  the  early  analyses.  The  information  might  also  be  used  to  automatically  generate 
documentation  and  nm-time  help. 

Another  ^roach  is  to  allow  the  designer  to  specify  the  design  in  an  appropriate  notation,  and  then 
provide  tools  to  convert  that  notation  into  interfaces.  For  examfde,  the  UAN  [26]  is  a  notation  for 
expressing  the  end  user’s  actions  and  the  system’s  responses. 
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Hnally,  much  work  is  needed  in  ways  for  tools  to  help  evaluate  interface  designs.  Initial  attempts,  such 
as  in  MIKE  [73],  have  highlighted  the  need  for  better  models  and  metrics  against  which  to  evaluate  the 
user  interfaces.  Research  in  this  area  is  continuing  by  cognitive  psychologists  and  other  user  interface 
researchers  (e.g.,  [12]). 

11.3  Increased  Breadth 

We  can  expect  the  user  interfaces  of  tomorrow  to  be  different  from  the  conventional  window-and- 
mouse  interfaces  of  today,  and  tools  will  have  to  change  to  support  the  new  styles.  For  example,  most 
tools  today  only  deal  with  two-dimensional  objects,  but  there  is  already  a  demand  to  provide  3-D 
visualizations  and  animations.  New  input  devices  and  techniques  will  probably  replace  the  conventional 
mouse  and  menu  styles.  For  example,  gesture  and  handwriting  recognition  are  appearing  in  mass-market 
commercial  products,  such  as  notepad  computers  and  ''personal  digital  assistants”  like  Apple’s  Newton 
(gesture  recognition  has  actually  ^  cn  used  since  the  1970s  in  commercial  CAD  tools).  ‘‘Virtual  reality” 
systems,  where  the  compute.  •  reates  an  artificial  world  and  allows  the  user  to  explore  it,  cannot  be 
handled  by  any  of  today’s  tools.  In  these  ‘‘non- WIMP”  applications  (WIMP  stands  for  Windows,  Icons, 
Menus  and  Pointing  devices),  designers  will  also  need  better  control  over  the  timing  of  the  interface,  to 
support  animations  and  various  new  media  like  video  [69].  Although  a  few  tools  are  directed  at  multiple- 
user  applications,  there  are  no  direct  graphical  specification  tools,  and  the  current  tools  arc  limited  the 
styles  of  applications  they  support. 

A  more  immediate  concern  is  supporting  interfaces  that  can  be  moved  from  one  natural  language  to 
another  (like  English  to  French).  Internationalizing  an  interface  is  much  more  difficult  than  simply 
translating  the  text  strings,  and  may  include  different  number,  date,  and  time  formats,  new  input  methods, 
redesigned  layouts,  different  color  schemes,  and  new  icons  [88].  How  can  future  tools  help  with  this 
process? 

11.4  End  User  Programming  and  Customization 

One  of  the  most  successful  computer  programs  of  all  time  is  the  spreadsheet  The  primary  reason  for 
its  success  is  that  end  users  can  program  (by  writing  formulas  and  macros).  However,  end  user 
programming  is  rare  in  other  applications,  and  where  it  exists,  usually  requires  learning  conventional 
programming.  For  example,  AutoCAD  provides  Lisp  for  customization.  More  effective  mechanisms  for 
users  to  customize  existing  applications  and  create  new  ones  are  needed  [61  ].  However,  ihc.se  should  not 
be  built  into  individual  applications  as  is  done  today,  siiKe  this  means  that  the  user  must  learn  a  different 
programming  technique  for  each  application.  Instead,  the  facilities  should  be  provided  at  the  system 
level,  arxl  therefore  should  be  part  of  the  underlying  toolkit  Naturally,  since  this  is  aimed  at  end  users,  it 
will  not  be  like  programming  in  C,  but  rather  at  some  higher  level. 
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The  X  Business  Group  predicts  that  there  will  be  an  increased  use  of  tools  by  end  users,  rather  than 
professional  software  developers,  which  wiU  present  enormous  opportunities  and  challenges  to  tool 
creators  [126], 

There  are  many  levels  at  which  users  might  want  to  modify  these  “malleable  interfaces:”  simple 
changing  of  menus  and  properties,  direct  programming  of  new  functions  like  in  spreadsheets,  or 
connecting  together  pre-built  components,  as  in  the  Andrew  and  OLE  frameworks.  Future  UI  tools 
should  support  changes  at  all  of  these  levels. 

11.5  Application  and  UI  Separation 

One  of  the  fundamental  goals  of  user  interface  tools  is  to  allow  the  better  modularization  and  separation 
of  user  interface  code  from  application  code.  However,  a  recent  survey  reported  that  modem  toolkits 
actually  make  this  separation  more  difficult,  due  to  the  large  number  of  call-back  procedures  required 
[57].  Therefore,  further  research  is  needed  into  ways  to  better  modularize  the  code,  and  how  tools  can 
support  this. 

11.6  Tools  for  the  Tools 

It  is  very  difficult  to  create  the  tools  described  in  this  paper.  Each  one  takes  an  enormous  effort. 
Therefore,  work  is  needed  in  ways  to  make  the  tools  themselves  easier  to  create.  For  example,  the  Garnet 
toolkit  is  exploring  mechanisms  specifically  designed  to  make  high-level  graphical  tools  easier  to  create 
[62].  The  Unidraw  framework  has  also  proven  useful  for  creating  interface  builders  [120].  However, 
more  work  is  needed. 

12.  Conclusions 

The  area  of  user  interface  tools  is  expanding  rapidly.  Five  years  ago,  you  would  have  been  hard- 
pressed  to  find  any  successful  commercial  higher-level  tools,  but  now  there  are  over  100  different  tools, 
and  tools  are  turning  into  a  billion  dollar  a  year  business.  Chances  are  that  today,  whatever  your  project 
is,  there  is  a  tool  that  will  help.  Tools  that  are  coming  out  of  research  labs  are  covering  increasingly  more 
of  the  user  interface  task,  are  more  effective  at  helping  the  designer,  and  are  creating  better  user 
interfaces.  As  more  companies  and  researchers  are  attracted  to  this  area,  we  can  expect  the  pace  of 
innovation  to  continue  to  accelerate.  There  will  be  many  exciting  and  aseful  new  tools  available  in  the 
near  future. 
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